Sep 17 13:14:50.980531 ip-10-0-133-247 systemd[1]: Starting Kubernetes Kubelet... Sep 17 13:14:51.376211 ip-10-0-133-247 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 13:14:51.376211 ip-10-0-133-247 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 13:14:51.376211 ip-10-0-133-247 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 13:14:51.376211 ip-10-0-133-247 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 13:14:51.376211 ip-10-0-133-247 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 13:14:51.376980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.376898 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 13:14:51.381916 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381894 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 13:14:51.381916 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381912 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 13:14:51.381916 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381916 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 13:14:51.381916 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381921 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381926 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381929 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381932 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381935 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381937 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381940 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381942 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381945 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381948 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381950 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381954 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381957 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381960 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381970 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381973 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381976 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381978 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381981 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381983 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 13:14:51.382078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381986 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381988 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381991 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381993 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381995 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.381998 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382000 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382002 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382005 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382007 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382011 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382013 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382016 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382018 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382021 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382023 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382026 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382028 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382030 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 13:14:51.382587 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382033 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382035 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382038 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382040 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382044 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382046 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382049 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382051 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382054 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382056 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382059 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382061 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382064 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382066 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382069 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382071 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382074 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382076 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382078 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382081 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 13:14:51.383040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382083 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382086 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382089 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382092 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382094 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382097 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382099 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382103 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382105 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382108 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382110 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382113 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382115 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382118 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382120 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382122 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382125 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382127 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382131 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382133 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 13:14:51.383531 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382136 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382139 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382141 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382144 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382146 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382148 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382151 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382153 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382156 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382158 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382160 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382163 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382165 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382168 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382170 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382173 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.382176 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383608 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383616 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 13:14:51.383994 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383619 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383623 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383626 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383628 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383632 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383635 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383639 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383641 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383644 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383646 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383649 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383651 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383654 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383656 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383659 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383662 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383665 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383668 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383670 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 13:14:51.384455 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383672 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383675 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383678 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383680 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383683 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383685 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383688 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383691 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383695 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383698 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383701 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383704 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383706 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383709 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383712 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383715 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383718 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383720 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383723 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383725 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 13:14:51.384912 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383728 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383730 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383732 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383735 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383737 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383739 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383741 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383744 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383747 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383750 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383752 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383755 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383757 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383759 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383762 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383764 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383766 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383769 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383771 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383774 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 13:14:51.385397 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383776 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383782 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383785 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383787 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383790 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383792 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383795 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383797 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383799 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383802 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383804 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383807 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383809 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383812 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383814 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383816 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383819 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383821 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383824 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383826 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 13:14:51.385878 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383829 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383831 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383834 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383837 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383839 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383845 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383848 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383851 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383854 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383856 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383859 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383862 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383864 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383867 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383870 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383873 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383875 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.383878 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384243 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 13:14:51.386371 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384248 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384251 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384254 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384257 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384260 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384263 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384266 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384284 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384288 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384291 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384294 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384296 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384299 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384302 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384305 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384308 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384311 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384313 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384316 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384318 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 13:14:51.386822 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384320 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384323 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384326 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384328 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384330 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384333 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384336 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384338 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384341 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384343 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384345 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384348 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384350 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384353 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384355 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384357 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384359 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384362 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384364 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 13:14:51.387294 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384367 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384369 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384371 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384374 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384376 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384378 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384381 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384383 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384386 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384388 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384391 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384393 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384396 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384398 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384400 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384402 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384406 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384409 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384414 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 13:14:51.387869 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384418 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384422 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384425 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384428 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384430 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384433 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384435 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384438 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384441 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384443 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384446 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384448 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384450 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384453 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384455 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384458 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384461 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384463 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384466 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384470 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 13:14:51.388353 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384472 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384474 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384477 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384480 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384482 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384484 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384487 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384489 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384492 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384494 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384497 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384499 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384501 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384504 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384506 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384509 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384511 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384514 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384516 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.384518 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384585 2575 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 13:14:51.388833 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384622 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384629 2575 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384633 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384637 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384640 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384644 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384647 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384650 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384653 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384656 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384659 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384662 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384665 2575 flags.go:64] FLAG: --cgroup-root="" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384667 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384670 2575 flags.go:64] FLAG: --client-ca-file="" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384674 2575 flags.go:64] FLAG: --cloud-provider="external" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384677 2575 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384681 2575 flags.go:64] FLAG: --cluster-domain="" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384684 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384686 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384689 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384692 2575 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384696 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 13:14:51.389376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384699 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384701 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384704 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384707 2575 flags.go:64] FLAG: --contention-profiling="false" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384710 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384712 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384715 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384718 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384721 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384724 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384727 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384729 2575 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384732 2575 flags.go:64] FLAG: --enable-server="true" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384735 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384742 2575 flags.go:64] FLAG: --event-burst="100" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384745 2575 flags.go:64] FLAG: --event-qps="50" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384747 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384750 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384753 2575 flags.go:64] FLAG: --eviction-hard="" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384758 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384761 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384764 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384767 2575 flags.go:64] FLAG: --eviction-soft="" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384769 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384773 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 13:14:51.389924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384777 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384780 2575 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384782 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384785 2575 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384788 2575 flags.go:64] FLAG: --feature-gates="" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384792 2575 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384794 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384797 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384800 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384803 2575 flags.go:64] FLAG: --healthz-port="10248" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384805 2575 flags.go:64] FLAG: --help="false" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384808 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384811 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384813 2575 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384816 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384819 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384823 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384826 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384828 2575 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384831 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384833 2575 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384836 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384839 2575 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384841 2575 flags.go:64] FLAG: --kube-reserved="" Sep 17 13:14:51.390575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384844 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384847 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384854 2575 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384857 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384859 2575 flags.go:64] FLAG: --lock-file="" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384862 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384865 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384869 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384874 2575 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384877 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384880 2575 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384883 2575 flags.go:64] FLAG: --logging-format="text" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384885 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384888 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384891 2575 flags.go:64] FLAG: --manifest-url="" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384893 2575 flags.go:64] FLAG: --manifest-url-header="" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384897 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384901 2575 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384904 2575 flags.go:64] FLAG: --max-pods="110" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384907 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384910 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384912 2575 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384916 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384919 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384922 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 13:14:51.391149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384925 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384931 2575 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384934 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384937 2575 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384940 2575 flags.go:64] FLAG: --pod-cidr="" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384942 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384948 2575 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384950 2575 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384953 2575 flags.go:64] FLAG: --pods-per-core="0" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384957 2575 flags.go:64] FLAG: --port="10250" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384960 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384963 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0565a205b61a8c564" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384966 2575 flags.go:64] FLAG: --qos-reserved="" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384968 2575 flags.go:64] FLAG: --read-only-port="10255" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384971 2575 flags.go:64] FLAG: --register-node="true" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384975 2575 flags.go:64] FLAG: --register-with-taints="" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384978 2575 flags.go:64] FLAG: --registry-burst="10" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384982 2575 flags.go:64] FLAG: --registry-qps="5" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384985 2575 flags.go:64] FLAG: --reserved-cpus="" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384987 2575 flags.go:64] FLAG: --reserved-memory="" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384991 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384993 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384996 2575 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.384999 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385002 2575 flags.go:64] FLAG: --runonce="false" Sep 17 13:14:51.391778 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385004 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385007 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385010 2575 flags.go:64] FLAG: --seccomp-default="false" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385013 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385015 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385018 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385021 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385024 2575 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385027 2575 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385029 2575 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385032 2575 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385034 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385037 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385040 2575 flags.go:64] FLAG: --system-cgroups="" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385043 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385047 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385051 2575 flags.go:64] FLAG: --tls-cert-file="" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385054 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385057 2575 flags.go:64] FLAG: --tls-min-version="" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385060 2575 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385063 2575 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385065 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385069 2575 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385072 2575 flags.go:64] FLAG: --v="2" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385075 2575 flags.go:64] FLAG: --version="false" Sep 17 13:14:51.392447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385079 2575 flags.go:64] FLAG: --vmodule="" Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385083 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.385086 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385193 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385198 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385201 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385204 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385206 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385209 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385212 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385214 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385217 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385220 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385222 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385225 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385227 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385230 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385232 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385235 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385237 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 13:14:51.393054 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385239 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385242 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385244 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385248 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385251 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385255 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385259 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385263 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385266 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385284 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385287 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385290 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385292 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385296 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385298 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385301 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385304 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385306 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385308 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385312 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 13:14:51.393547 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385316 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385319 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385322 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385324 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385327 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385329 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385332 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385335 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385337 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385339 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385342 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385345 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385347 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385350 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385352 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385356 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385358 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385360 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385363 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385365 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 13:14:51.394014 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385368 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385371 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385374 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385376 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385379 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385381 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385384 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385386 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385389 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385391 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385394 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385396 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385399 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385401 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385405 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385409 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385412 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385415 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385418 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 13:14:51.394504 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385421 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385423 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385425 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385428 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385430 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385433 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385435 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385438 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385442 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385446 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385448 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385451 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385454 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385456 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385460 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385462 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385465 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385467 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385470 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 13:14:51.394961 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385472 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385475 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385477 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.385480 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.386013 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.393055 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.393070 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.394142 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.394149 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394207 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394212 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394215 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394219 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394221 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394224 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 13:14:51.395427 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394227 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394229 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394232 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394234 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394237 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394240 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394242 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394245 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394248 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394250 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394253 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394255 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394257 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394260 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394262 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394265 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394267 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394285 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394289 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 13:14:51.395812 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394292 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394294 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394298 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394302 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394307 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394310 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394312 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394315 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394317 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394320 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394322 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394325 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394327 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394330 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394332 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394335 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394337 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394340 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394342 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394345 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 13:14:51.396262 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394347 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394349 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394352 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394354 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394356 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394359 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394361 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394364 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394366 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394368 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394371 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394373 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394375 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394378 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394380 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394383 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394386 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394389 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394392 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394394 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 13:14:51.396820 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394396 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394399 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394401 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394404 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394406 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394409 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394411 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394413 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394416 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394418 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394420 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394423 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394426 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394428 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394430 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394433 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394435 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394438 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394440 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394443 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 13:14:51.397307 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394445 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394447 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394450 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394452 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394454 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394457 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394459 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394462 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394466 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394469 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394472 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394475 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394478 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394481 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 13:14:51.397788 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.394486 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394592 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394597 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394600 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394603 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394606 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394609 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394612 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394615 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394617 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394620 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394622 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394625 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394627 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394630 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394632 2575 feature_gate.go:328] unrecognized feature gate: Example Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394635 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394637 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394639 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 13:14:51.398116 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394642 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394644 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394647 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394649 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394652 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394654 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394656 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394659 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394662 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394664 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394667 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394670 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394672 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394675 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394677 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394680 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394682 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394684 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394687 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 13:14:51.398586 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394689 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394691 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394694 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394696 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394698 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394701 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394703 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394705 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394708 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394710 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394713 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394715 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394717 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394720 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394722 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394724 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394727 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394729 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394732 2575 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394734 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394736 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 13:14:51.399021 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394739 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394741 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394744 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394747 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394749 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394752 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394754 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394757 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394759 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394761 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394764 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394766 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394768 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394771 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394773 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394776 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394778 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394780 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394784 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 13:14:51.399539 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394787 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394790 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394792 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394795 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394797 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394800 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394802 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394805 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394809 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394811 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394814 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394816 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394819 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394821 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394824 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394826 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394829 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394832 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394834 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394838 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 13:14:51.399983 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394841 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 13:14:51.400457 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:51.394844 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 13:14:51.400457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.394849 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 13:14:51.400457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.394944 2575 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 13:14:51.400457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.397683 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 13:14:51.400457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.398595 2575 server.go:1019] "Starting client certificate rotation" Sep 17 13:14:51.400457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.398692 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 13:14:51.400457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.398721 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 13:14:51.401098 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.401085 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 13:14:51.406914 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.406895 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 13:14:51.417619 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.417603 2575 log.go:25] "Validated CRI v1 runtime API" Sep 17 13:14:51.420340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.420328 2575 log.go:25] "Validated CRI v1 image API" Sep 17 13:14:51.421132 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.421118 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 13:14:51.424179 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.424157 2575 fs.go:135] Filesystem UUIDs: map[2e10d274-2346-450a-8eab-5ce80f82ad62:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ae00c9ee-9888-4859-9637-6cb6744e0652:/dev/nvme0n1p3] Sep 17 13:14:51.424254 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.424177 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 13:14:51.425492 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.425476 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 13:14:51.428944 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.428848 2575 manager.go:216] Machine: {Timestamp:2026-09-17 13:14:51.42769741 +0000 UTC m=+0.332266094 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3226694 MemoryCapacity:32812146688 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a0c71bc489045197b9dad2720e625 SystemUUID:ec2a0c71-bc48-9045-197b-9dad2720e625 BootID:ef8cfb2c-89ff-4364-ba2e-804397042bc7 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406073344 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fa:30:7f:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fa:30:7f:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:54:90:86:62:10 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812146688 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 13:14:51.428944 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.428941 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 13:14:51.429052 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.429012 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 13:14:51.431961 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.431936 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 13:14:51.432082 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.431963 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-247.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 13:14:51.433734 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.433724 2575 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 13:14:51.433765 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.433737 2575 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 13:14:51.433765 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.433745 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 13:14:51.434429 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.434419 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 13:14:51.435266 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.435255 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 13:14:51.439243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439230 2575 state_mem.go:36] "Initialized new in-memory state store" Sep 17 13:14:51.439408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439398 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 13:14:51.439440 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439426 2575 kubelet.go:486] "Attempting to sync node with API server" Sep 17 13:14:51.439440 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439435 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 13:14:51.439491 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439445 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 13:14:51.439491 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439454 2575 kubelet.go:398] "Adding apiserver pod source" Sep 17 13:14:51.439491 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439463 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 13:14:51.439816 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439802 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 13:14:51.439864 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.439831 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 13:14:51.442063 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.442031 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 17 13:14:51.444602 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.444584 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 13:14:51.444661 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.444616 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 13:14:51.446507 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446491 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 13:14:51.446507 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446509 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446516 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446525 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446534 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446539 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446552 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446557 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446568 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446574 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446582 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 13:14:51.446612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.446586 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 13:14:51.447368 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.447358 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 13:14:51.447368 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.447367 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 13:14:51.449434 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.449390 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 13:14:51.449524 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.449506 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-247.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 13:14:51.449608 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.449523 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-247.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 13:14:51.451961 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.451947 2575 server.go:1268] "Started kubelet" Sep 17 13:14:51.452031 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.452010 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 13:14:51.452088 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.452027 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 13:14:51.452088 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.452078 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 13:14:51.452735 ip-10-0-133-247 systemd[1]: Started Kubernetes Kubelet. Sep 17 13:14:51.453551 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.453537 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 13:14:51.453783 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.453682 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w5kcd" Sep 17 13:14:51.454130 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.454116 2575 server.go:310] "Adding debug handlers to kubelet server" Sep 17 13:14:51.456001 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.455978 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 13:14:51.460104 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.460090 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 13:14:51.460643 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.460630 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 13:14:51.460809 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.460786 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:51.461449 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461430 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 13:14:51.461449 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461434 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 13:14:51.461583 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461460 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 13:14:51.461583 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461567 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w5kcd" Sep 17 13:14:51.461583 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461577 2575 reconstruct.go:97] "Volume reconstruction finished" Sep 17 13:14:51.461583 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461585 2575 reconciler.go:29] "Reconciler: start to sync state" Sep 17 13:14:51.461697 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461601 2575 factory.go:55] Registering systemd factory Sep 17 13:14:51.461697 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461614 2575 factory.go:223] Registration of the systemd container factory successfully Sep 17 13:14:51.462050 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461887 2575 factory.go:153] Registering CRI-O factory Sep 17 13:14:51.462050 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461903 2575 factory.go:223] Registration of the crio container factory successfully Sep 17 13:14:51.462050 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461953 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 13:14:51.462050 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461972 2575 factory.go:103] Registering Raw factory Sep 17 13:14:51.462050 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.461987 2575 manager.go:1196] Started watching for new ooms in manager Sep 17 13:14:51.463102 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.463087 2575 manager.go:349] Starting recovery of all containers Sep 17 13:14:51.466520 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.466481 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 13:14:51.466790 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.466764 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-247.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 13:14:51.467782 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.466626 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-247.ec2.internal.18d61e00f8f3c6f9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-247.ec2.internal,UID:ip-10-0-133-247.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-247.ec2.internal,},FirstTimestamp:2026-09-17 13:14:51.451909881 +0000 UTC m=+0.356478548,LastTimestamp:2026-09-17 13:14:51.451909881 +0000 UTC m=+0.356478548,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-247.ec2.internal,}" Sep 17 13:14:51.473856 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.473832 2575 manager.go:354] Recovery completed Sep 17 13:14:51.478457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.478442 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 13:14:51.480452 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.480437 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientMemory" Sep 17 13:14:51.480533 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.480468 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 13:14:51.480533 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.480482 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientPID" Sep 17 13:14:51.480978 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.480966 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 13:14:51.480978 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.480977 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 13:14:51.481064 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.480984 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 13:14:51.481064 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.480992 2575 state_mem.go:36] "Initialized new in-memory state store" Sep 17 13:14:51.481655 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.481640 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientMemory" Sep 17 13:14:51.481726 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.481666 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 13:14:51.481726 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.481676 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientPID" Sep 17 13:14:51.482967 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.482957 2575 policy_none.go:49] "None policy: Start" Sep 17 13:14:51.483006 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.482971 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 13:14:51.483006 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.482980 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 13:14:51.484938 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.484928 2575 policy_none.go:47] "Start" Sep 17 13:14:51.530973 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.530844 2575 manager.go:337] "Starting Device Plugin manager" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.531015 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.531028 2575 server.go:87] "Starting device plugin registration server" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.531218 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.531231 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.531355 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.531426 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.531435 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.531861 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 13:14:51.542893 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.531903 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:51.600770 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.600750 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 13:14:51.602037 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.602023 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 13:14:51.602087 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.602073 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 13:14:51.602123 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.602096 2575 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 13:14:51.602172 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.602138 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 13:14:51.604482 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.604464 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 13:14:51.631642 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.631593 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 13:14:51.632350 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.632335 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientMemory" Sep 17 13:14:51.632401 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.632363 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 13:14:51.632401 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.632374 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientPID" Sep 17 13:14:51.632401 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.632395 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.638885 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.638872 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.638936 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.638892 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-247.ec2.internal\": node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:51.652340 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.652321 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:51.703016 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.702994 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal"] Sep 17 13:14:51.703094 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.703050 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 13:14:51.704354 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.704337 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientMemory" Sep 17 13:14:51.704426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.704363 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 13:14:51.704426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.704378 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientPID" Sep 17 13:14:51.706624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.706613 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 13:14:51.706753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.706738 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.706809 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.706766 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 13:14:51.707252 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.707235 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientMemory" Sep 17 13:14:51.707358 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.707255 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 13:14:51.707358 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.707290 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientPID" Sep 17 13:14:51.707435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.707360 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientMemory" Sep 17 13:14:51.707435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.707382 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 13:14:51.707435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.707392 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientPID" Sep 17 13:14:51.709546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.709530 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.709601 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.709559 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 13:14:51.710206 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.710183 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientMemory" Sep 17 13:14:51.710206 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.710207 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 13:14:51.710338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.710222 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeHasSufficientPID" Sep 17 13:14:51.731734 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.731708 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-247.ec2.internal\" not found" node="ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.735840 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.735826 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-247.ec2.internal\" not found" node="ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.752531 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.752514 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:51.763377 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.763357 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f0b88c1b4c38f740f3c86482bf822bb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal\" (UID: \"2f0b88c1b4c38f740f3c86482bf822bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.763448 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.763379 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f0b88c1b4c38f740f3c86482bf822bb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal\" (UID: \"2f0b88c1b4c38f740f3c86482bf822bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.763448 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.763404 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ad660f1163c376144183c1081f4e9aa-config\") pod \"kube-apiserver-proxy-ip-10-0-133-247.ec2.internal\" (UID: \"7ad660f1163c376144183c1081f4e9aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.852597 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.852552 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:51.863921 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.863900 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f0b88c1b4c38f740f3c86482bf822bb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal\" (UID: \"2f0b88c1b4c38f740f3c86482bf822bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.864001 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.863950 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2f0b88c1b4c38f740f3c86482bf822bb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal\" (UID: \"2f0b88c1b4c38f740f3c86482bf822bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.864056 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.863999 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f0b88c1b4c38f740f3c86482bf822bb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal\" (UID: \"2f0b88c1b4c38f740f3c86482bf822bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.864056 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.864030 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ad660f1163c376144183c1081f4e9aa-config\") pod \"kube-apiserver-proxy-ip-10-0-133-247.ec2.internal\" (UID: \"7ad660f1163c376144183c1081f4e9aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.864136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.864062 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2f0b88c1b4c38f740f3c86482bf822bb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal\" (UID: \"2f0b88c1b4c38f740f3c86482bf822bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.864136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:51.864067 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ad660f1163c376144183c1081f4e9aa-config\") pod \"kube-apiserver-proxy-ip-10-0-133-247.ec2.internal\" (UID: \"7ad660f1163c376144183c1081f4e9aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" Sep 17 13:14:51.953230 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:51.953176 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.034700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.034668 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:52.039605 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.039368 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" Sep 17 13:14:52.054093 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.054070 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.154705 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.154662 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.255196 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.255114 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.355740 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.355701 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.398160 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.398136 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 13:14:52.398550 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.398264 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 13:14:52.455996 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.455974 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.466479 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.466453 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 13:09:52 +0000 UTC" deadline="2028-06-05 21:39:11.828777194 +0000 UTC" Sep 17 13:14:52.466479 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.466476 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15056h24m19.362304191s" Sep 17 13:14:52.491784 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.491766 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 13:14:52.556351 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.556319 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.646478 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:52.646449 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad660f1163c376144183c1081f4e9aa.slice/crio-f19caeafffa47a0f221aa2d2612192985b057ad1602ed09f42f63dc44e7b3a84 WatchSource:0}: Error finding container f19caeafffa47a0f221aa2d2612192985b057ad1602ed09f42f63dc44e7b3a84: Status 404 returned error can't find the container with id f19caeafffa47a0f221aa2d2612192985b057ad1602ed09f42f63dc44e7b3a84 Sep 17 13:14:52.649969 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.649956 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 13:14:52.657130 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.657115 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.757671 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.757636 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.791064 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.791007 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 13:14:52.795772 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:52.795752 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f0b88c1b4c38f740f3c86482bf822bb.slice/crio-da8b6d825a022650526b1670f4758dc3a979114aeb67d34020561d4569d76c13 WatchSource:0}: Error finding container da8b6d825a022650526b1670f4758dc3a979114aeb67d34020561d4569d76c13: Status 404 returned error can't find the container with id da8b6d825a022650526b1670f4758dc3a979114aeb67d34020561d4569d76c13 Sep 17 13:14:52.858560 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.858538 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.959048 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:52.959017 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-247.ec2.internal\" not found" Sep 17 13:14:52.973649 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:52.973625 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 13:14:53.064067 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.063993 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" Sep 17 13:14:53.078836 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.078815 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 13:14:53.079735 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.079722 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" Sep 17 13:14:53.088313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.088274 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 13:14:53.440926 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.440840 2575 apiserver.go:52] "Watching apiserver" Sep 17 13:14:53.450868 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.450840 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 13:14:53.451339 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.451290 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-p4p6z","kube-system/konnectivity-agent-qvpbl","kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wpz88","openshift-multus/multus-8ndwf","openshift-multus/network-metrics-daemon-lnvpk","openshift-network-operator/iptables-alerter-rb8tb","openshift-ovn-kubernetes/ovnkube-node-wh9bs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6","openshift-dns/node-resolver-qmw76","openshift-image-registry/node-ca-6tpjm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal","openshift-multus/multus-additional-cni-plugins-pn7zx","openshift-network-diagnostics/network-check-target-2f8lr"] Sep 17 13:14:53.454144 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.454118 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.456396 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.456374 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.456594 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.456568 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 13:14:53.456866 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.456844 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 13:14:53.456940 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.456893 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 13:14:53.457118 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.457099 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j6xc8\"" Sep 17 13:14:53.458995 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.458786 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:53.458995 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.458857 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:14:53.459510 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.459486 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 13:14:53.459735 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.459719 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dc5gj\"" Sep 17 13:14:53.459817 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.459788 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 13:14:53.459942 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.459925 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 13:14:53.460023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.460007 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 13:14:53.460226 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.460210 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 13:14:53.460774 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.460756 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 13:14:53.461287 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.461260 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.464079 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.464009 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.464171 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.464114 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 13:14:53.464249 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.464228 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xtfcg\"" Sep 17 13:14:53.464730 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.464484 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 13:14:53.466084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.466048 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fhlt5\"" Sep 17 13:14:53.466174 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.466118 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 13:14:53.468907 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.466434 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:53.468907 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.466509 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:14:53.468907 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.468878 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.471551 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.471252 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.471551 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.471261 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 13:14:53.471551 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.471324 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:14:53.471551 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.471326 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 13:14:53.471551 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.471464 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wqh7l\"" Sep 17 13:14:53.471858 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.471587 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.474572 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475025 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-modprobe-d\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475053 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-lib-modules\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475071 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-system-cni-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475086 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97385e5c-6b79-4d4f-9a52-49aace78ffaf-cni-binary-copy\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475109 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-kubelet\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2flh\" (UniqueName: \"kubernetes.io/projected/97385e5c-6b79-4d4f-9a52-49aace78ffaf-kube-api-access-r2flh\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475162 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmxsd\" (UniqueName: \"kubernetes.io/projected/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-kube-api-access-vmxsd\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475189 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-sys\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475210 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-hostroot\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475233 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02945a85-1d01-43ae-ab66-57114b5b8a0e-host-slash\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475257 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475280 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-var-lib-kubelet\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475390 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-tuned\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475495 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-etc-kubernetes\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475523 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.475823 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475549 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysconfig\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475578 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-kubernetes\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475600 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysctl-d\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475641 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-systemd\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475685 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b10f405-2e76-4f8c-a846-987bf8f131e8-tmp\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475711 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-socket-dir-parent\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475754 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-daemon-config\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475788 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-system-cni-dir\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475811 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-run\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475835 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-cnibin\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475860 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-k8s-cni-cncf-io\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475900 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-cni-multus\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475934 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.475974 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-cni-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476044 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-netns\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.483578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476071 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-multus-certs\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cnibin\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476119 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476142 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-cni-bin\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476167 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02945a85-1d01-43ae-ab66-57114b5b8a0e-iptables-alerter-script\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476193 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6clq8\" (UniqueName: \"kubernetes.io/projected/02945a85-1d01-43ae-ab66-57114b5b8a0e-kube-api-access-6clq8\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476225 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5f4p\" (UniqueName: \"kubernetes.io/projected/1b10f405-2e76-4f8c-a846-987bf8f131e8-kube-api-access-j5f4p\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476257 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysctl-conf\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476274 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-host\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476332 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-conf-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476538 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xql4j\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.476775 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.477044 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.477084 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.477333 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.479089 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.479399 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nzdpd\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.479642 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.479684 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.481489 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.481591 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.481675 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.481761 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.481840 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7pgt2\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.483418 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.483784 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 13:14:53.484338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.483974 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 13:14:53.485612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.484152 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 13:14:53.485612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.484758 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 13:14:53.485612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.484996 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6bcz2\"" Sep 17 13:14:53.485612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.485059 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 13:14:53.497455 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.497430 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ptvjl" Sep 17 13:14:53.506109 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.506088 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ptvjl" Sep 17 13:14:53.562851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.562829 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 13:14:53.577370 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577346 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-host\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.577485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-conf-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.577485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577412 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-device-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.577485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577438 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-etc-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.577485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577442 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-host\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.577485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-conf-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.577485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577463 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-env-overrides\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.577770 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577502 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-lib-modules\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.577770 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97385e5c-6b79-4d4f-9a52-49aace78ffaf-cni-binary-copy\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.577770 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577680 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-lib-modules\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.577770 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577696 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmxsd\" (UniqueName: \"kubernetes.io/projected/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-kube-api-access-vmxsd\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.577770 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577735 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02945a85-1d01-43ae-ab66-57114b5b8a0e-host-slash\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.577770 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577765 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-ovn\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577789 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-node-log\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577817 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w27h7\" (UniqueName: \"kubernetes.io/projected/bcccedee-c77d-4596-a8b4-f304f9c38c13-kube-api-access-w27h7\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577843 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-var-lib-kubelet\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577895 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-tuned\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577920 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-etc-kubernetes\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577946 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-hosts-file\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-cni-netd\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.577991 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02945a85-1d01-43ae-ab66-57114b5b8a0e-host-slash\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578033 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovn-node-metrics-cert\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.578084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578060 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578083 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-kubernetes\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578122 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b10f405-2e76-4f8c-a846-987bf8f131e8-tmp\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578148 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-socket-dir-parent\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-daemon-config\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578185 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97385e5c-6b79-4d4f-9a52-49aace78ffaf-cni-binary-copy\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578199 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-kubelet-config\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578226 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7d54d52-b7fa-4cc0-a27b-4da60b500985-host\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578227 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578271 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578324 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-etc-kubernetes\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.578688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578438 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-socket-dir-parent\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578887 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578934 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-run\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578945 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-daemon-config\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.578975 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579005 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1844561-1e55-48e3-aee5-a2af535b8003-konnectivity-ca\") pod \"konnectivity-agent-qvpbl\" (UID: \"b1844561-1e55-48e3-aee5-a2af535b8003\") " pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579033 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-sys-fs\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579060 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvxqz\" (UniqueName: \"kubernetes.io/projected/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-kube-api-access-nvxqz\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579094 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-kubelet\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579119 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-run-netns\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579147 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579175 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-cni-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579204 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-netns\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.579225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579228 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-slash\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579254 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cnibin\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579376 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6clq8\" (UniqueName: \"kubernetes.io/projected/02945a85-1d01-43ae-ab66-57114b5b8a0e-kube-api-access-6clq8\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579408 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-registration-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579459 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-cni-bin\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579481 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovnkube-config\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579506 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5f4p\" (UniqueName: \"kubernetes.io/projected/1b10f405-2e76-4f8c-a846-987bf8f131e8-kube-api-access-j5f4p\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579566 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysctl-conf\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579594 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1844561-1e55-48e3-aee5-a2af535b8003-agent-certs\") pod \"konnectivity-agent-qvpbl\" (UID: \"b1844561-1e55-48e3-aee5-a2af535b8003\") " pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579619 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579645 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e7d54d52-b7fa-4cc0-a27b-4da60b500985-serviceca\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-modprobe-d\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-system-cni-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579731 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-kubelet\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579756 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579855 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2flh\" (UniqueName: \"kubernetes.io/projected/97385e5c-6b79-4d4f-9a52-49aace78ffaf-kube-api-access-r2flh\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.579883 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579885 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-socket-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579918 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-sys\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579957 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-hostroot\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.579988 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77f2h\" (UniqueName: \"kubernetes.io/projected/e7d54d52-b7fa-4cc0-a27b-4da60b500985-kube-api-access-77f2h\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580025 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-systemd\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580044 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-kubernetes\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580050 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-log-socket\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580091 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-kubelet\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-systemd-units\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580149 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysconfig\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580177 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysctl-d\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580203 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-systemd\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580233 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-run-ovn-kubernetes\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580273 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580305 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-system-cni-dir\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580339 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-cnibin\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580387 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-k8s-cni-cncf-io\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.580744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580405 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysctl-conf\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580416 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-cni-multus\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580464 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-cni-multus\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580502 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-cnibin\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580508 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-modprobe-d\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580550 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-run\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580595 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-system-cni-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580775 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-sys\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580825 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-hostroot\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gtgf\" (UniqueName: \"kubernetes.io/projected/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-kube-api-access-4gtgf\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580934 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.580965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581008 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-multus-certs\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581038 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blm7r\" (UniqueName: \"kubernetes.io/projected/9e063f7d-b387-4af5-944e-d77ffefceb18-kube-api-access-blm7r\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581067 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581111 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-tmp-dir\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581142 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-var-lib-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.581641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581199 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-cni-bin\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581204 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-multus-cni-dir\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581230 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02945a85-1d01-43ae-ab66-57114b5b8a0e-iptables-alerter-script\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581258 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-dbus\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581264 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-netns\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581286 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovnkube-script-lib\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581456 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-system-cni-dir\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581500 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-multus-certs\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581546 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-systemd\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581554 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysconfig\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581562 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-cnibin\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581587 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-var-lib-kubelet\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581593 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581623 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-run-k8s-cni-cncf-io\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581653 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/97385e5c-6b79-4d4f-9a52-49aace78ffaf-host-var-lib-cni-bin\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.581655 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-sysctl-d\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.582285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.582164 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02945a85-1d01-43ae-ab66-57114b5b8a0e-iptables-alerter-script\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.585653 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.585604 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1b10f405-2e76-4f8c-a846-987bf8f131e8-etc-tuned\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.589559 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.588765 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b10f405-2e76-4f8c-a846-987bf8f131e8-tmp\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.590265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.589865 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmxsd\" (UniqueName: \"kubernetes.io/projected/9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b-kube-api-access-vmxsd\") pod \"multus-additional-cni-plugins-pn7zx\" (UID: \"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b\") " pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.590265 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.589870 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 13:14:53.590265 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.589938 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 13:14:53.590265 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.589953 2575 projected.go:196] Error preparing data for projected volume kube-api-access-9kq6t for pod openshift-network-diagnostics/network-check-target-2f8lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:14:53.590265 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.590060 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t podName:cca57ec8-2a52-4896-a70d-fdd82708f7f4 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:54.090038714 +0000 UTC m=+2.994607391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9kq6t" (UniqueName: "kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t") pod "network-check-target-2f8lr" (UID: "cca57ec8-2a52-4896-a70d-fdd82708f7f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:14:53.592892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.592850 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6clq8\" (UniqueName: \"kubernetes.io/projected/02945a85-1d01-43ae-ab66-57114b5b8a0e-kube-api-access-6clq8\") pod \"iptables-alerter-rb8tb\" (UID: \"02945a85-1d01-43ae-ab66-57114b5b8a0e\") " pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.594449 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.594427 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2flh\" (UniqueName: \"kubernetes.io/projected/97385e5c-6b79-4d4f-9a52-49aace78ffaf-kube-api-access-r2flh\") pod \"multus-8ndwf\" (UID: \"97385e5c-6b79-4d4f-9a52-49aace78ffaf\") " pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.595879 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.595434 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5f4p\" (UniqueName: \"kubernetes.io/projected/1b10f405-2e76-4f8c-a846-987bf8f131e8-kube-api-access-j5f4p\") pod \"tuned-wpz88\" (UID: \"1b10f405-2e76-4f8c-a846-987bf8f131e8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.607683 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.607635 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" event={"ID":"2f0b88c1b4c38f740f3c86482bf822bb","Type":"ContainerStarted","Data":"da8b6d825a022650526b1670f4758dc3a979114aeb67d34020561d4569d76c13"} Sep 17 13:14:53.609127 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.609102 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" event={"ID":"7ad660f1163c376144183c1081f4e9aa","Type":"ContainerStarted","Data":"f19caeafffa47a0f221aa2d2612192985b057ad1602ed09f42f63dc44e7b3a84"} Sep 17 13:14:53.681967 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.681917 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-registration-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.682129 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.681981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.682129 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.682005 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-cni-bin\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.682129 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.682034 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovnkube-config\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.687424 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687392 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1844561-1e55-48e3-aee5-a2af535b8003-agent-certs\") pod \"konnectivity-agent-qvpbl\" (UID: \"b1844561-1e55-48e3-aee5-a2af535b8003\") " pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:14:53.687559 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.687559 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687498 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e7d54d52-b7fa-4cc0-a27b-4da60b500985-serviceca\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.687559 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687545 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-socket-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.687684 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77f2h\" (UniqueName: \"kubernetes.io/projected/e7d54d52-b7fa-4cc0-a27b-4da60b500985-kube-api-access-77f2h\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.687684 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-systemd\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.687684 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687642 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-log-socket\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.687684 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-systemd-units\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.687831 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-run-ovn-kubernetes\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.687831 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687752 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.687831 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687799 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gtgf\" (UniqueName: \"kubernetes.io/projected/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-kube-api-access-4gtgf\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.687963 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687861 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.687963 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687889 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blm7r\" (UniqueName: \"kubernetes.io/projected/9e063f7d-b387-4af5-944e-d77ffefceb18-kube-api-access-blm7r\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:53.687963 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687930 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-tmp-dir\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.688106 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.687972 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-var-lib-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688106 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688021 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-dbus\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.688106 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688053 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovnkube-script-lib\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688194 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688148 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-device-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.688228 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-etc-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688228 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688216 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-env-overrides\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688284 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688249 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-ovn\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688316 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688290 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-node-log\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688371 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w27h7\" (UniqueName: \"kubernetes.io/projected/bcccedee-c77d-4596-a8b4-f304f9c38c13-kube-api-access-w27h7\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688410 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688399 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-hosts-file\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.688468 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688455 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-cni-netd\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688513 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688489 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovn-node-metrics-cert\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-kubelet-config\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.688603 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688587 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7d54d52-b7fa-4cc0-a27b-4da60b500985-host\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.688654 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688625 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688690 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688673 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:53.688723 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688706 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1844561-1e55-48e3-aee5-a2af535b8003-konnectivity-ca\") pod \"konnectivity-agent-qvpbl\" (UID: \"b1844561-1e55-48e3-aee5-a2af535b8003\") " pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:14:53.688768 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688737 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-sys-fs\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.688818 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688794 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvxqz\" (UniqueName: \"kubernetes.io/projected/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-kube-api-access-nvxqz\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.688857 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-kubelet\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688890 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688860 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-run-netns\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.688935 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.688910 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-slash\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.689057 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.689044 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-slash\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.689687 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.689663 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.689777 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.689726 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-cni-bin\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.689777 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.689768 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-registration-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.690060 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.689828 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-device-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.690060 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.689940 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 13:14:53.690060 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.690033 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret podName:ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:54.190013344 +0000 UTC m=+3.094582020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret") pod "global-pull-secret-syncer-p4p6z" (UID: "ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084") : object "kube-system"/"original-pull-secret" not registered Sep 17 13:14:53.690808 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.690763 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-socket-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.690908 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.690834 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e7d54d52-b7fa-4cc0-a27b-4da60b500985-serviceca\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.691002 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.690928 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.691068 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.690994 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-kubelet-config\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.691132 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-etc-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691132 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691097 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-systemd\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691242 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691211 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-dbus\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:53.691293 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691272 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-log-socket\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691318 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-systemd-units\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691416 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691397 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-run-ovn-kubernetes\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691476 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691650 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691621 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-env-overrides\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691799 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691780 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-tmp-dir\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.691846 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691807 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovnkube-script-lib\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691888 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691847 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-ovn\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691925 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691899 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7d54d52-b7fa-4cc0-a27b-4da60b500985-host\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.691956 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691945 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-run-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.691988 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691973 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-sys-fs\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.692029 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691986 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-kubelet\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.692029 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.691997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovnkube-config\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.692096 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.692029 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-run-netns\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.692096 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.692066 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:14:53.692096 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.692088 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-hosts-file\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.692191 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:53.692131 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs podName:9e063f7d-b387-4af5-944e-d77ffefceb18 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:54.192102357 +0000 UTC m=+3.096671032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs") pod "network-metrics-daemon-lnvpk" (UID: "9e063f7d-b387-4af5-944e-d77ffefceb18") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:14:53.692191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.692133 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-node-log\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.692191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.692183 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-host-cni-netd\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.692292 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.692238 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bcccedee-c77d-4596-a8b4-f304f9c38c13-var-lib-openvswitch\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.692411 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.692365 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b1844561-1e55-48e3-aee5-a2af535b8003-konnectivity-ca\") pod \"konnectivity-agent-qvpbl\" (UID: \"b1844561-1e55-48e3-aee5-a2af535b8003\") " pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:14:53.693628 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.693603 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b1844561-1e55-48e3-aee5-a2af535b8003-agent-certs\") pod \"konnectivity-agent-qvpbl\" (UID: \"b1844561-1e55-48e3-aee5-a2af535b8003\") " pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:14:53.695365 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.695321 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bcccedee-c77d-4596-a8b4-f304f9c38c13-ovn-node-metrics-cert\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.699666 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.699629 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gtgf\" (UniqueName: \"kubernetes.io/projected/3319af9c-edac-4f9e-8e6c-9bc9feb432ec-kube-api-access-4gtgf\") pod \"node-resolver-qmw76\" (UID: \"3319af9c-edac-4f9e-8e6c-9bc9feb432ec\") " pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.701225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.701198 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blm7r\" (UniqueName: \"kubernetes.io/projected/9e063f7d-b387-4af5-944e-d77ffefceb18-kube-api-access-blm7r\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:53.702035 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.702008 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77f2h\" (UniqueName: \"kubernetes.io/projected/e7d54d52-b7fa-4cc0-a27b-4da60b500985-kube-api-access-77f2h\") pod \"node-ca-6tpjm\" (UID: \"e7d54d52-b7fa-4cc0-a27b-4da60b500985\") " pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.702570 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.702548 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvxqz\" (UniqueName: \"kubernetes.io/projected/9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce-kube-api-access-nvxqz\") pod \"aws-ebs-csi-driver-node-bsrz6\" (UID: \"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.703522 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.703488 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w27h7\" (UniqueName: \"kubernetes.io/projected/bcccedee-c77d-4596-a8b4-f304f9c38c13-kube-api-access-w27h7\") pod \"ovnkube-node-wh9bs\" (UID: \"bcccedee-c77d-4596-a8b4-f304f9c38c13\") " pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:53.770638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.770596 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rb8tb" Sep 17 13:14:53.787147 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.787125 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" Sep 17 13:14:53.796926 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:53.796897 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1d3d91_1e42_4ef4_8f2c_addd5949ce8b.slice/crio-388793a93dcb94274c2d3d548dc0761df4ff7c543f0566a572763303539c916d WatchSource:0}: Error finding container 388793a93dcb94274c2d3d548dc0761df4ff7c543f0566a572763303539c916d: Status 404 returned error can't find the container with id 388793a93dcb94274c2d3d548dc0761df4ff7c543f0566a572763303539c916d Sep 17 13:14:53.798905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.798885 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wpz88" Sep 17 13:14:53.807567 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.807542 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8ndwf" Sep 17 13:14:53.808778 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:53.808752 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b10f405_2e76_4f8c_a846_987bf8f131e8.slice/crio-9d3c6b82837fc1ebebd3faf768a9fe261b59122bebe3bbe93f36facfe7ad134c WatchSource:0}: Error finding container 9d3c6b82837fc1ebebd3faf768a9fe261b59122bebe3bbe93f36facfe7ad134c: Status 404 returned error can't find the container with id 9d3c6b82837fc1ebebd3faf768a9fe261b59122bebe3bbe93f36facfe7ad134c Sep 17 13:14:53.814800 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.814781 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 13:14:53.815570 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.815399 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" Sep 17 13:14:53.825352 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.825324 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:14:53.833200 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.833179 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qmw76" Sep 17 13:14:53.842467 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.842441 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6tpjm" Sep 17 13:14:53.851660 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:53.851580 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:14:54.092024 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:54.091936 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:54.092235 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:54.092086 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 13:14:54.092235 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:54.092110 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 13:14:54.092235 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:54.092121 2575 projected.go:196] Error preparing data for projected volume kube-api-access-9kq6t for pod openshift-network-diagnostics/network-check-target-2f8lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:14:54.092235 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:54.092185 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t podName:cca57ec8-2a52-4896-a70d-fdd82708f7f4 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:55.092165862 +0000 UTC m=+3.996734543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9kq6t" (UniqueName: "kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t") pod "network-check-target-2f8lr" (UID: "cca57ec8-2a52-4896-a70d-fdd82708f7f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:14:54.192748 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:54.192720 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:54.192911 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:54.192759 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:54.192911 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:54.192867 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 13:14:54.192911 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:54.192898 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:14:54.193029 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:54.192933 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret podName:ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:55.192914474 +0000 UTC m=+4.097483152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret") pod "global-pull-secret-syncer-p4p6z" (UID: "ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084") : object "kube-system"/"original-pull-secret" not registered Sep 17 13:14:54.193029 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:54.192973 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs podName:9e063f7d-b387-4af5-944e-d77ffefceb18 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:55.192946837 +0000 UTC m=+4.097515500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs") pod "network-metrics-daemon-lnvpk" (UID: "9e063f7d-b387-4af5-944e-d77ffefceb18") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:14:54.509850 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:54.509807 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 13:09:54 +0000 UTC" deadline="2028-06-02 15:26:31.49576598 +0000 UTC" Sep 17 13:14:54.509850 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:54.509845 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14978h11m36.985926321s" Sep 17 13:14:54.612359 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:54.612239 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wpz88" event={"ID":"1b10f405-2e76-4f8c-a846-987bf8f131e8","Type":"ContainerStarted","Data":"9d3c6b82837fc1ebebd3faf768a9fe261b59122bebe3bbe93f36facfe7ad134c"} Sep 17 13:14:54.613719 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:54.613662 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerStarted","Data":"388793a93dcb94274c2d3d548dc0761df4ff7c543f0566a572763303539c916d"} Sep 17 13:14:54.615206 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:54.615175 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rb8tb" event={"ID":"02945a85-1d01-43ae-ab66-57114b5b8a0e","Type":"ContainerStarted","Data":"e53bbd1786f10de0ab0e1e2b64997185f46476f0fd25610bff962ccff84aa180"} Sep 17 13:14:55.101261 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.101229 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:55.101548 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.101422 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 13:14:55.101548 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.101450 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 13:14:55.101548 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.101464 2575 projected.go:196] Error preparing data for projected volume kube-api-access-9kq6t for pod openshift-network-diagnostics/network-check-target-2f8lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:14:55.101548 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.101528 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t podName:cca57ec8-2a52-4896-a70d-fdd82708f7f4 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:57.101509689 +0000 UTC m=+6.006078363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9kq6t" (UniqueName: "kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t") pod "network-check-target-2f8lr" (UID: "cca57ec8-2a52-4896-a70d-fdd82708f7f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:14:55.160630 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:55.160598 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcccedee_c77d_4596_a8b4_f304f9c38c13.slice/crio-30587d3d9f1e6e6fe382b96c197b574759211bde77387e368ec7599874ae1edc WatchSource:0}: Error finding container 30587d3d9f1e6e6fe382b96c197b574759211bde77387e368ec7599874ae1edc: Status 404 returned error can't find the container with id 30587d3d9f1e6e6fe382b96c197b574759211bde77387e368ec7599874ae1edc Sep 17 13:14:55.161010 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:55.160986 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d54d52_b7fa_4cc0_a27b_4da60b500985.slice/crio-798e0a99ae0c48abdb5f4770e21296a38aab350f58f25156dd65570d4095b0ae WatchSource:0}: Error finding container 798e0a99ae0c48abdb5f4770e21296a38aab350f58f25156dd65570d4095b0ae: Status 404 returned error can't find the container with id 798e0a99ae0c48abdb5f4770e21296a38aab350f58f25156dd65570d4095b0ae Sep 17 13:14:55.163864 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:55.163841 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb7a2d5_7bb7_4705_8d7b_a7f291ca32ce.slice/crio-182fb4f98f24209437bbd75e64b6161c2fca52710d64f7073b56850bd4ec7a57 WatchSource:0}: Error finding container 182fb4f98f24209437bbd75e64b6161c2fca52710d64f7073b56850bd4ec7a57: Status 404 returned error can't find the container with id 182fb4f98f24209437bbd75e64b6161c2fca52710d64f7073b56850bd4ec7a57 Sep 17 13:14:55.166810 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:55.166750 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3319af9c_edac_4f9e_8e6c_9bc9feb432ec.slice/crio-5c6019d34dcdf7eb31c204a8d2ffa739546b0dc99e29ca69a42252123d5dcb43 WatchSource:0}: Error finding container 5c6019d34dcdf7eb31c204a8d2ffa739546b0dc99e29ca69a42252123d5dcb43: Status 404 returned error can't find the container with id 5c6019d34dcdf7eb31c204a8d2ffa739546b0dc99e29ca69a42252123d5dcb43 Sep 17 13:14:55.169045 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:14:55.169018 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1844561_1e55_48e3_aee5_a2af535b8003.slice/crio-e846ba255adc4ec3ac54c868ea960e7725ef973001f7decba06c73165877d8c7 WatchSource:0}: Error finding container e846ba255adc4ec3ac54c868ea960e7725ef973001f7decba06c73165877d8c7: Status 404 returned error can't find the container with id e846ba255adc4ec3ac54c868ea960e7725ef973001f7decba06c73165877d8c7 Sep 17 13:14:55.204969 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.202675 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:55.204969 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.202768 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:55.204969 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.202889 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:14:55.204969 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.202941 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs podName:9e063f7d-b387-4af5-944e-d77ffefceb18 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:57.2029232 +0000 UTC m=+6.107491862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs") pod "network-metrics-daemon-lnvpk" (UID: "9e063f7d-b387-4af5-944e-d77ffefceb18") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:14:55.204969 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.203275 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 13:14:55.204969 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.203325 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret podName:ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084 nodeName:}" failed. No retries permitted until 2026-09-17 13:14:57.203308839 +0000 UTC m=+6.107877498 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret") pod "global-pull-secret-syncer-p4p6z" (UID: "ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084") : object "kube-system"/"original-pull-secret" not registered Sep 17 13:14:55.510668 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.510340 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 13:09:54 +0000 UTC" deadline="2028-05-12 21:00:27.746126886 +0000 UTC" Sep 17 13:14:55.510668 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.510585 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14479h45m32.235548044s" Sep 17 13:14:55.602877 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.602847 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:55.602999 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.602974 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:14:55.603218 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.603198 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:55.603342 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.603307 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:14:55.603342 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.603313 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:55.603476 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:55.603446 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:14:55.619553 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.619522 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8ndwf" event={"ID":"97385e5c-6b79-4d4f-9a52-49aace78ffaf","Type":"ContainerStarted","Data":"8d50e188f9f8cf86e48b447501c80a0f70b99ea28d6c09946c9ffc15c88137b9"} Sep 17 13:14:55.621140 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.620930 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qmw76" event={"ID":"3319af9c-edac-4f9e-8e6c-9bc9feb432ec","Type":"ContainerStarted","Data":"5c6019d34dcdf7eb31c204a8d2ffa739546b0dc99e29ca69a42252123d5dcb43"} Sep 17 13:14:55.622475 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.622449 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6tpjm" event={"ID":"e7d54d52-b7fa-4cc0-a27b-4da60b500985","Type":"ContainerStarted","Data":"798e0a99ae0c48abdb5f4770e21296a38aab350f58f25156dd65570d4095b0ae"} Sep 17 13:14:55.624867 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.624839 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" event={"ID":"7ad660f1163c376144183c1081f4e9aa","Type":"ContainerStarted","Data":"fe6961e607652e61e7c63576d95032bd5484a595e36604dd881565b3dea75b5c"} Sep 17 13:14:55.626478 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.626456 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvpbl" event={"ID":"b1844561-1e55-48e3-aee5-a2af535b8003","Type":"ContainerStarted","Data":"e846ba255adc4ec3ac54c868ea960e7725ef973001f7decba06c73165877d8c7"} Sep 17 13:14:55.628594 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.628568 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"30587d3d9f1e6e6fe382b96c197b574759211bde77387e368ec7599874ae1edc"} Sep 17 13:14:55.630127 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.630101 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" event={"ID":"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce","Type":"ContainerStarted","Data":"182fb4f98f24209437bbd75e64b6161c2fca52710d64f7073b56850bd4ec7a57"} Sep 17 13:14:55.631929 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.631905 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" event={"ID":"2f0b88c1b4c38f740f3c86482bf822bb","Type":"ContainerStarted","Data":"65e64afdb0a8b770f8df8caa7833b4a761b833d07f913bcddb8159badba570f0"} Sep 17 13:14:55.638612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:55.638567 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-247.ec2.internal" podStartSLOduration=2.6385554730000003 podStartE2EDuration="2.638555473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:14:55.638555174 +0000 UTC m=+4.543123854" watchObservedRunningTime="2026-09-17 13:14:55.638555473 +0000 UTC m=+4.543124153" Sep 17 13:14:56.639314 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:56.639199 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f0b88c1b4c38f740f3c86482bf822bb" containerID="65e64afdb0a8b770f8df8caa7833b4a761b833d07f913bcddb8159badba570f0" exitCode=0 Sep 17 13:14:56.640112 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:56.640083 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" event={"ID":"2f0b88c1b4c38f740f3c86482bf822bb","Type":"ContainerDied","Data":"65e64afdb0a8b770f8df8caa7833b4a761b833d07f913bcddb8159badba570f0"} Sep 17 13:14:57.117763 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:57.117658 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:57.117976 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.117908 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 13:14:57.117976 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.117932 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 13:14:57.117976 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.117947 2575 projected.go:196] Error preparing data for projected volume kube-api-access-9kq6t for pod openshift-network-diagnostics/network-check-target-2f8lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:14:57.118136 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.118008 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t podName:cca57ec8-2a52-4896-a70d-fdd82708f7f4 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:01.117988396 +0000 UTC m=+10.022557070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9kq6t" (UniqueName: "kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t") pod "network-check-target-2f8lr" (UID: "cca57ec8-2a52-4896-a70d-fdd82708f7f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:14:57.219072 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:57.219016 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:57.219283 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:57.219081 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:57.219283 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.219220 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 13:14:57.219283 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.219282 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret podName:ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:01.219264494 +0000 UTC m=+10.123833175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret") pod "global-pull-secret-syncer-p4p6z" (UID: "ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084") : object "kube-system"/"original-pull-secret" not registered Sep 17 13:14:57.219973 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.219718 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:14:57.219973 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.219789 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs podName:9e063f7d-b387-4af5-944e-d77ffefceb18 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:01.219771506 +0000 UTC m=+10.124340179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs") pod "network-metrics-daemon-lnvpk" (UID: "9e063f7d-b387-4af5-944e-d77ffefceb18") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:14:57.603516 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:57.603435 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:57.603516 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:57.603457 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:57.603758 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.603566 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:14:57.603758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:57.603630 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:57.603874 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.603775 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:14:57.603874 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:57.603858 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:14:59.603333 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:59.603119 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:14:59.603333 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:59.603146 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:14:59.603333 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:14:59.603118 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:14:59.603333 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:59.603266 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:14:59.603995 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:59.603437 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:14:59.603995 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:14:59.603533 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:01.154578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:01.154046 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:01.154578 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.154189 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 13:15:01.154578 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.154206 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 13:15:01.154578 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.154217 2575 projected.go:196] Error preparing data for projected volume kube-api-access-9kq6t for pod openshift-network-diagnostics/network-check-target-2f8lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:15:01.154578 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.154262 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t podName:cca57ec8-2a52-4896-a70d-fdd82708f7f4 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:09.154248589 +0000 UTC m=+18.058817260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9kq6t" (UniqueName: "kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t") pod "network-check-target-2f8lr" (UID: "cca57ec8-2a52-4896-a70d-fdd82708f7f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:15:01.255121 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:01.255087 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:01.255323 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:01.255146 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:01.255323 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.255298 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 13:15:01.255431 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.255379 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret podName:ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:09.255358999 +0000 UTC m=+18.159927674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret") pod "global-pull-secret-syncer-p4p6z" (UID: "ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084") : object "kube-system"/"original-pull-secret" not registered Sep 17 13:15:01.255431 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.255401 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:15:01.255558 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.255439 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs podName:9e063f7d-b387-4af5-944e-d77ffefceb18 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:09.255426508 +0000 UTC m=+18.159995181 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs") pod "network-metrics-daemon-lnvpk" (UID: "9e063f7d-b387-4af5-944e-d77ffefceb18") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:15:01.603200 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:01.603123 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:01.603393 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.603239 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:01.603644 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:01.603624 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:01.603753 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.603719 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:01.603827 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:01.603783 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:01.603884 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:01.603871 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:03.603299 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:03.603199 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:03.603702 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:03.603338 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:03.603755 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:03.603712 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:03.606104 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:03.603803 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:03.606104 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:03.603865 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:03.606104 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:03.603959 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:05.603884 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:05.603698 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:05.603884 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:05.603690 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:05.603884 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:05.603879 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:05.604444 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:05.604150 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:05.604444 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:05.604316 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:05.604444 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:05.604414 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:07.602768 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:07.602745 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:07.603105 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:07.602772 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:07.603105 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:07.602745 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:07.603105 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:07.602870 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:07.603105 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:07.602954 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:07.603105 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:07.603050 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:09.216263 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:09.216229 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:09.216730 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.216410 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 13:15:09.216730 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.216435 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 13:15:09.216730 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.216449 2575 projected.go:196] Error preparing data for projected volume kube-api-access-9kq6t for pod openshift-network-diagnostics/network-check-target-2f8lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:15:09.216730 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.216507 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t podName:cca57ec8-2a52-4896-a70d-fdd82708f7f4 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:25.216491638 +0000 UTC m=+34.121060293 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9kq6t" (UniqueName: "kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t") pod "network-check-target-2f8lr" (UID: "cca57ec8-2a52-4896-a70d-fdd82708f7f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:15:09.317420 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:09.317382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:09.317583 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:09.317463 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:09.317583 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.317549 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:15:09.317583 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.317551 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 13:15:09.317747 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.317609 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs podName:9e063f7d-b387-4af5-944e-d77ffefceb18 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:25.317588778 +0000 UTC m=+34.222157453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs") pod "network-metrics-daemon-lnvpk" (UID: "9e063f7d-b387-4af5-944e-d77ffefceb18") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 13:15:09.317747 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.317622 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret podName:ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:25.317616442 +0000 UTC m=+34.222185098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret") pod "global-pull-secret-syncer-p4p6z" (UID: "ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084") : object "kube-system"/"original-pull-secret" not registered Sep 17 13:15:09.602990 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:09.602910 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:09.603153 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:09.602910 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:09.603153 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.603040 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:09.603153 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.603129 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:09.603153 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:09.602914 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:09.603361 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:09.603231 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:11.604313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.602773 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:11.604313 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:11.602893 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:11.604313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.603299 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:11.604313 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:11.603394 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:11.604792 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.604403 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:11.604792 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:11.604507 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:11.683862 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.683081 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" event={"ID":"2f0b88c1b4c38f740f3c86482bf822bb","Type":"ContainerStarted","Data":"9f34c476d67b9b5066395c4f17634edc432578f50790819b96aa1696636f255b"} Sep 17 13:15:11.692152 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.691629 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wpz88" event={"ID":"1b10f405-2e76-4f8c-a846-987bf8f131e8","Type":"ContainerStarted","Data":"7da7c374806afb571aecd70bc49d562f4c09612a5803a5fe040915dfedbe8bc9"} Sep 17 13:15:11.697240 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.697206 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerStarted","Data":"93c84bc784df3b616a055535b4b6992c924a9e4706d931a410bb2ca34745f3e7"} Sep 17 13:15:11.699671 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.698984 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-247.ec2.internal" podStartSLOduration=18.698966911 podStartE2EDuration="18.698966911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:11.697926351 +0000 UTC m=+20.602495029" watchObservedRunningTime="2026-09-17 13:15:11.698966911 +0000 UTC m=+20.603535591" Sep 17 13:15:11.711915 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.711890 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qmw76" event={"ID":"3319af9c-edac-4f9e-8e6c-9bc9feb432ec","Type":"ContainerStarted","Data":"1cac1bed482abef9c773d582041dded59efb53f423256afd76f642e2aa771f39"} Sep 17 13:15:11.734710 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.734669 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wpz88" podStartSLOduration=2.080905682 podStartE2EDuration="19.734655788s" totalImagesPullingTime="17.653750106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:11.73424325 +0000 UTC m=+20.638811944" watchObservedRunningTime="2026-09-17 13:15:11.734655788 +0000 UTC m=+20.639224464" Sep 17 13:15:11.747228 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:11.747194 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qmw76" podStartSLOduration=7.320811294 podStartE2EDuration="19.747180788s" totalImagesPullingTime="12.426369494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:11.747068498 +0000 UTC m=+20.651637175" watchObservedRunningTime="2026-09-17 13:15:11.747180788 +0000 UTC m=+20.651749464" Sep 17 13:15:12.715179 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.715004 2575 generic.go:356] "Generic (PLEG): container finished" podID="9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b" containerID="93c84bc784df3b616a055535b4b6992c924a9e4706d931a410bb2ca34745f3e7" exitCode=0 Sep 17 13:15:12.715978 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.715090 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerDied","Data":"93c84bc784df3b616a055535b4b6992c924a9e4706d931a410bb2ca34745f3e7"} Sep 17 13:15:12.716893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.716849 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8ndwf" event={"ID":"97385e5c-6b79-4d4f-9a52-49aace78ffaf","Type":"ContainerStarted","Data":"dbf0d03f7c93214e592fe530b902a51d5d429eb1d5ef1b77a4ef131f732dfeb7"} Sep 17 13:15:12.718350 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.718329 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6tpjm" event={"ID":"e7d54d52-b7fa-4cc0-a27b-4da60b500985","Type":"ContainerStarted","Data":"c4621738b4291d26672b1af4f8f7a496e7ffdb6d207b16b1036b6c59ce28ade6"} Sep 17 13:15:12.719804 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.719782 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rb8tb" event={"ID":"02945a85-1d01-43ae-ab66-57114b5b8a0e","Type":"ContainerStarted","Data":"12170509e67ff3bdb591d34a039a6db87cb1fe08ee8cf55b9e3ce947c66a1141"} Sep 17 13:15:12.721301 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.721262 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qvpbl" event={"ID":"b1844561-1e55-48e3-aee5-a2af535b8003","Type":"ContainerStarted","Data":"5b594eee5b1a5502ef4b43a327a33cf3a78a83288b94a059a3cb56e2882ed02d"} Sep 17 13:15:12.723799 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.723785 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:15:12.724104 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.724086 2575 generic.go:356] "Generic (PLEG): container finished" podID="bcccedee-c77d-4596-a8b4-f304f9c38c13" containerID="2431612d243f60672d9ad5382642ab9c064a39b46a46401c596a37f2e4adad14" exitCode=1 Sep 17 13:15:12.724184 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.724141 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"05bfc488a971490249e637705b6dd25be169ce38624e1a7ec9be4bc0370a0529"} Sep 17 13:15:12.724228 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.724195 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"a1b4a3ad08e781e7850bdb5cd2f14db5eaa69ce430544e8758e54230e53d17dd"} Sep 17 13:15:12.724287 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.724232 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"562214f9c7a1e3746f443b07c223d48ffff67a299fa274156c49dee808dba6c8"} Sep 17 13:15:12.724326 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.724291 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"031aacada03f096ad642b826008dc73a9fc1286019dbc1b73b37dd7ce0524926"} Sep 17 13:15:12.724326 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.724321 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerDied","Data":"2431612d243f60672d9ad5382642ab9c064a39b46a46401c596a37f2e4adad14"} Sep 17 13:15:12.724410 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.724350 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"468974ead01973f7a16564f833ad298d893ec853e23d58f391f9b542a06daf17"} Sep 17 13:15:12.725568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.725544 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" event={"ID":"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce","Type":"ContainerStarted","Data":"f4ede8011bf9b85b12b686c214c60c22fce53596a86dc182bf2ba2784ff15a63"} Sep 17 13:15:12.747300 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.747247 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8ndwf" podStartSLOduration=4.387993107 podStartE2EDuration="20.747237348s" totalImagesPullingTime="16.359244241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:12.746960814 +0000 UTC m=+21.651529501" watchObservedRunningTime="2026-09-17 13:15:12.747237348 +0000 UTC m=+21.651806025" Sep 17 13:15:12.759413 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.759357 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qvpbl" podStartSLOduration=4.491580806 podStartE2EDuration="20.759345884s" totalImagesPullingTime="16.267765078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:12.759292142 +0000 UTC m=+21.663860812" watchObservedRunningTime="2026-09-17 13:15:12.759345884 +0000 UTC m=+21.663914562" Sep 17 13:15:12.793154 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:12.793113 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6tpjm" podStartSLOduration=4.493004419 podStartE2EDuration="20.793101848s" totalImagesPullingTime="16.300097429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:12.778100683 +0000 UTC m=+21.682669360" watchObservedRunningTime="2026-09-17 13:15:12.793101848 +0000 UTC m=+21.697670525" Sep 17 13:15:13.603027 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:13.603004 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:13.603102 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:13.603004 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:13.603155 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:13.603105 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:13.603155 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:13.603108 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:13.603233 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:13.603164 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:13.603267 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:13.603232 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:13.709115 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:13.709093 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 13:15:13.729750 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:13.729677 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" event={"ID":"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce","Type":"ContainerStarted","Data":"09ab8a96cb181c378270f05ab5c53bcf71698a137226cefa483a62dcf90db510"} Sep 17 13:15:14.544714 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:14.544548 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T13:15:13.709112293Z","UUID":"5a2b23b5-d41d-495a-b10a-84be2e1d8ddc","Handler":null,"Name":"","Endpoint":""} Sep 17 13:15:14.546236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:14.546212 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 13:15:14.546236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:14.546249 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 13:15:14.734561 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:14.734538 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:15:14.734933 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:14.734858 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"ed347904c9452bc949aa4b3c9c5be63edb24363dd0ff159f62047cacf72f0d97"} Sep 17 13:15:15.603147 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:15.602920 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:15.603342 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:15.602925 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:15.603342 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:15.603231 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:15.603342 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:15.603301 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:15.603342 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:15.602968 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:15.603515 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:15.603374 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:15.740078 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:15.740035 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" event={"ID":"9eb7a2d5-7bb7-4705-8d7b-a7f291ca32ce","Type":"ContainerStarted","Data":"bf14fab8c20b388da9e033d8f8740bbbcd4d2f7ee1341c8c15c3a63151cbe43c"} Sep 17 13:15:15.760287 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:15.760221 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rb8tb" podStartSLOduration=14.363630816 podStartE2EDuration="23.760203851s" totalImagesPullingTime="9.396573035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:12.793157721 +0000 UTC m=+21.697726397" watchObservedRunningTime="2026-09-17 13:15:15.760203851 +0000 UTC m=+24.664772530" Sep 17 13:15:15.760532 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:15.760486 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bsrz6" podStartSLOduration=4.640732978 podStartE2EDuration="23.760475902s" totalImagesPullingTime="19.119742924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:15.76005161 +0000 UTC m=+24.664620310" watchObservedRunningTime="2026-09-17 13:15:15.760475902 +0000 UTC m=+24.665044579" Sep 17 13:15:16.707075 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:16.706979 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:15:16.707928 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:16.707906 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:15:16.746180 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:16.746148 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:15:16.747059 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:16.746589 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"7a9ed285e8293cdbf9bbe042d8caa03a8e19fbafcfd508b8151c0ef5076300e8"} Sep 17 13:15:16.747121 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:16.747081 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:15:16.747171 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:16.747145 2575 scope.go:117] "RemoveContainer" containerID="2431612d243f60672d9ad5382642ab9c064a39b46a46401c596a37f2e4adad14" Sep 17 13:15:16.747404 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:16.747389 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qvpbl" Sep 17 13:15:17.602962 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.602753 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:17.603132 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.602770 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:17.603132 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:17.603058 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:17.603132 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.602800 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:17.603304 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:17.603126 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:17.603304 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:17.603215 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:17.752520 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.752496 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:15:17.752937 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.752898 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" event={"ID":"bcccedee-c77d-4596-a8b4-f304f9c38c13","Type":"ContainerStarted","Data":"6b8b696bb114461b1c6568df62e1e7c865dd3a9090c7bd9280e1d040167ce98c"} Sep 17 13:15:17.753688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.753661 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:15:17.753786 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.753743 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:15:17.753851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.753788 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:15:17.769537 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.769512 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:15:17.769652 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.769640 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:15:17.781194 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:17.781147 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" podStartSLOduration=9.439792094 podStartE2EDuration="25.781129679s" totalImagesPullingTime="16.341337585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:17.780647605 +0000 UTC m=+26.685216304" watchObservedRunningTime="2026-09-17 13:15:17.781129679 +0000 UTC m=+26.685698360" Sep 17 13:15:19.607247 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:19.607223 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:19.607588 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:19.607331 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:19.607588 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:19.607337 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:19.607588 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:19.607356 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:19.607588 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:19.607426 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:19.607588 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:19.607488 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:19.761661 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:19.760408 2575 generic.go:356] "Generic (PLEG): container finished" podID="9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b" containerID="a4c51e0205738f4e7a438452335aa61654654c86d35fcb097ce6624430cd0056" exitCode=0 Sep 17 13:15:19.761661 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:19.761584 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerDied","Data":"a4c51e0205738f4e7a438452335aa61654654c86d35fcb097ce6624430cd0056"} Sep 17 13:15:20.398494 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:20.398408 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lnvpk"] Sep 17 13:15:20.398643 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:20.398529 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:20.398694 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:20.398653 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:20.398997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:20.398972 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p4p6z"] Sep 17 13:15:20.399093 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:20.399060 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:20.399160 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:20.399144 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:20.409536 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:20.409515 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2f8lr"] Sep 17 13:15:20.409661 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:20.409589 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:20.409723 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:20.409662 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:21.603657 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:21.603498 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:21.604001 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:21.603707 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:21.604001 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:21.603527 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:21.604001 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:21.603826 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:21.766901 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:21.766870 2575 generic.go:356] "Generic (PLEG): container finished" podID="9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b" containerID="b30adcb8dfe6d4e91988102b0ae03bc8fb89f3189f6516cd608f5b1e26bc2293" exitCode=0 Sep 17 13:15:21.767026 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:21.766926 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerDied","Data":"b30adcb8dfe6d4e91988102b0ae03bc8fb89f3189f6516cd608f5b1e26bc2293"} Sep 17 13:15:22.602588 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:22.602560 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:22.602704 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:22.602663 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnvpk" podUID="9e063f7d-b387-4af5-944e-d77ffefceb18" Sep 17 13:15:22.771107 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:22.771026 2575 generic.go:356] "Generic (PLEG): container finished" podID="9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b" containerID="f311f653e9ee967cf67a2cf7b77448cf87ed4c96db4502063957797ae51052a3" exitCode=0 Sep 17 13:15:22.771107 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:22.771076 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerDied","Data":"f311f653e9ee967cf67a2cf7b77448cf87ed4c96db4502063957797ae51052a3"} Sep 17 13:15:23.602471 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:23.602440 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:23.602792 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:23.602566 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2f8lr" podUID="cca57ec8-2a52-4896-a70d-fdd82708f7f4" Sep 17 13:15:23.602792 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:23.602625 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:23.602792 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:23.602723 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4p6z" podUID="ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084" Sep 17 13:15:24.344839 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.344812 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-247.ec2.internal" event="NodeReady" Sep 17 13:15:24.345255 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.344918 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 13:15:24.393923 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.393891 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r9d4n"] Sep 17 13:15:24.396695 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.396673 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.397108 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.397080 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g8f5x"] Sep 17 13:15:24.399126 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.399048 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 13:15:24.399126 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.399107 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 13:15:24.399317 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.399151 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hvcd4\"" Sep 17 13:15:24.400981 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.399798 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.402400 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.401919 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 13:15:24.402400 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.402256 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 13:15:24.402400 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.402265 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fv2wn\"" Sep 17 13:15:24.402595 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.402456 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 13:15:24.402595 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.402541 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 13:15:24.406523 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.406506 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r9d4n"] Sep 17 13:15:24.409487 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.409423 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g8f5x"] Sep 17 13:15:24.497469 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.497438 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9lbh4"] Sep 17 13:15:24.500380 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.500354 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9lbh4" Sep 17 13:15:24.502503 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.502476 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 13:15:24.502678 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.502646 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mkpd6\"" Sep 17 13:15:24.502800 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.502690 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 13:15:24.502800 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.502712 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 13:15:24.508630 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.507971 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9lbh4"] Sep 17 13:15:24.540525 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540501 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/111aa84c-189b-489c-b075-226fee25b80d-data-volume\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.540641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540539 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/644b480b-08f1-4fe9-b3db-177aaa8bcf46-metrics-tls\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.540641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540555 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/644b480b-08f1-4fe9-b3db-177aaa8bcf46-tmp-dir\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.540641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540575 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/111aa84c-189b-489c-b075-226fee25b80d-crio-socket\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.540641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540597 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/111aa84c-189b-489c-b075-226fee25b80d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.540838 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540679 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/111aa84c-189b-489c-b075-226fee25b80d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.540838 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/644b480b-08f1-4fe9-b3db-177aaa8bcf46-config-volume\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.540838 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540735 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmfsp\" (UniqueName: \"kubernetes.io/projected/644b480b-08f1-4fe9-b3db-177aaa8bcf46-kube-api-access-qmfsp\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.540838 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.540764 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7db2p\" (UniqueName: \"kubernetes.io/projected/111aa84c-189b-489c-b075-226fee25b80d-kube-api-access-7db2p\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.602753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.602727 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:24.604955 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.604927 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ksjdf\"" Sep 17 13:15:24.605071 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.604994 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 13:15:24.641744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641723 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/111aa84c-189b-489c-b075-226fee25b80d-crio-socket\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.641908 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641751 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/111aa84c-189b-489c-b075-226fee25b80d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.641908 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641779 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/111aa84c-189b-489c-b075-226fee25b80d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.641908 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641795 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/644b480b-08f1-4fe9-b3db-177aaa8bcf46-config-volume\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.641908 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641818 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmfsp\" (UniqueName: \"kubernetes.io/projected/644b480b-08f1-4fe9-b3db-177aaa8bcf46-kube-api-access-qmfsp\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.641908 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7db2p\" (UniqueName: \"kubernetes.io/projected/111aa84c-189b-489c-b075-226fee25b80d-kube-api-access-7db2p\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.641908 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641875 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nqwt\" (UniqueName: \"kubernetes.io/projected/acaee4a3-6596-4bdf-83e3-438894602170-kube-api-access-4nqwt\") pod \"ingress-canary-9lbh4\" (UID: \"acaee4a3-6596-4bdf-83e3-438894602170\") " pod="openshift-ingress-canary/ingress-canary-9lbh4" Sep 17 13:15:24.642191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641923 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acaee4a3-6596-4bdf-83e3-438894602170-cert\") pod \"ingress-canary-9lbh4\" (UID: \"acaee4a3-6596-4bdf-83e3-438894602170\") " pod="openshift-ingress-canary/ingress-canary-9lbh4" Sep 17 13:15:24.642191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641982 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/111aa84c-189b-489c-b075-226fee25b80d-data-volume\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.642191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.642024 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/644b480b-08f1-4fe9-b3db-177aaa8bcf46-metrics-tls\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.642191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.641984 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/111aa84c-189b-489c-b075-226fee25b80d-crio-socket\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.642191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.642051 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/644b480b-08f1-4fe9-b3db-177aaa8bcf46-tmp-dir\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.642426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.642324 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/644b480b-08f1-4fe9-b3db-177aaa8bcf46-tmp-dir\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.642426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.642337 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/111aa84c-189b-489c-b075-226fee25b80d-data-volume\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.642426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.642413 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/111aa84c-189b-489c-b075-226fee25b80d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.642513 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.642429 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/644b480b-08f1-4fe9-b3db-177aaa8bcf46-config-volume\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.645767 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.645747 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/644b480b-08f1-4fe9-b3db-177aaa8bcf46-metrics-tls\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.645767 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.645759 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/111aa84c-189b-489c-b075-226fee25b80d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.650171 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.650152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7db2p\" (UniqueName: \"kubernetes.io/projected/111aa84c-189b-489c-b075-226fee25b80d-kube-api-access-7db2p\") pod \"insights-runtime-extractor-g8f5x\" (UID: \"111aa84c-189b-489c-b075-226fee25b80d\") " pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.650371 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.650354 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmfsp\" (UniqueName: \"kubernetes.io/projected/644b480b-08f1-4fe9-b3db-177aaa8bcf46-kube-api-access-qmfsp\") pod \"dns-default-r9d4n\" (UID: \"644b480b-08f1-4fe9-b3db-177aaa8bcf46\") " pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.709548 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.709455 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:24.714558 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.714535 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g8f5x" Sep 17 13:15:24.742706 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.742665 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nqwt\" (UniqueName: \"kubernetes.io/projected/acaee4a3-6596-4bdf-83e3-438894602170-kube-api-access-4nqwt\") pod \"ingress-canary-9lbh4\" (UID: \"acaee4a3-6596-4bdf-83e3-438894602170\") " pod="openshift-ingress-canary/ingress-canary-9lbh4" Sep 17 13:15:24.742836 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.742733 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acaee4a3-6596-4bdf-83e3-438894602170-cert\") pod \"ingress-canary-9lbh4\" (UID: \"acaee4a3-6596-4bdf-83e3-438894602170\") " pod="openshift-ingress-canary/ingress-canary-9lbh4" Sep 17 13:15:24.745680 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.745656 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acaee4a3-6596-4bdf-83e3-438894602170-cert\") pod \"ingress-canary-9lbh4\" (UID: \"acaee4a3-6596-4bdf-83e3-438894602170\") " pod="openshift-ingress-canary/ingress-canary-9lbh4" Sep 17 13:15:24.753625 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.753598 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nqwt\" (UniqueName: \"kubernetes.io/projected/acaee4a3-6596-4bdf-83e3-438894602170-kube-api-access-4nqwt\") pod \"ingress-canary-9lbh4\" (UID: \"acaee4a3-6596-4bdf-83e3-438894602170\") " pod="openshift-ingress-canary/ingress-canary-9lbh4" Sep 17 13:15:24.811249 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:24.811221 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9lbh4" Sep 17 13:15:25.007658 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.007338 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g8f5x"] Sep 17 13:15:25.011410 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.011387 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r9d4n"] Sep 17 13:15:25.012706 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:25.012685 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111aa84c_189b_489c_b075_226fee25b80d.slice/crio-5a661c178abe0ecbacd6a77a2d708cc2d5cc8682b0fe905f6161c17527c1e5bb WatchSource:0}: Error finding container 5a661c178abe0ecbacd6a77a2d708cc2d5cc8682b0fe905f6161c17527c1e5bb: Status 404 returned error can't find the container with id 5a661c178abe0ecbacd6a77a2d708cc2d5cc8682b0fe905f6161c17527c1e5bb Sep 17 13:15:25.014421 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:25.014399 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644b480b_08f1_4fe9_b3db_177aaa8bcf46.slice/crio-395ec0f7f9f08ac7d6656b4fc7784c0a8d480114907529f594082ffb3446cc0d WatchSource:0}: Error finding container 395ec0f7f9f08ac7d6656b4fc7784c0a8d480114907529f594082ffb3446cc0d: Status 404 returned error can't find the container with id 395ec0f7f9f08ac7d6656b4fc7784c0a8d480114907529f594082ffb3446cc0d Sep 17 13:15:25.083725 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.083700 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9lbh4"] Sep 17 13:15:25.088426 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:25.088385 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacaee4a3_6596_4bdf_83e3_438894602170.slice/crio-8a1ee98667893f4d0688f33eaab414f744fe704fbec45f017d1c90d07ecbadef WatchSource:0}: Error finding container 8a1ee98667893f4d0688f33eaab414f744fe704fbec45f017d1c90d07ecbadef: Status 404 returned error can't find the container with id 8a1ee98667893f4d0688f33eaab414f744fe704fbec45f017d1c90d07ecbadef Sep 17 13:15:25.248466 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.248436 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:25.248649 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:25.248628 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 13:15:25.248712 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:25.248654 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 13:15:25.248712 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:25.248667 2575 projected.go:196] Error preparing data for projected volume kube-api-access-9kq6t for pod openshift-network-diagnostics/network-check-target-2f8lr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:15:25.248816 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:25.248726 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t podName:cca57ec8-2a52-4896-a70d-fdd82708f7f4 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:57.248705789 +0000 UTC m=+66.153274464 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9kq6t" (UniqueName: "kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t") pod "network-check-target-2f8lr" (UID: "cca57ec8-2a52-4896-a70d-fdd82708f7f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 13:15:25.349369 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.349284 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:25.349369 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.349343 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:25.350173 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:25.349449 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 13:15:25.350173 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:25.349504 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret podName:ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084 nodeName:}" failed. No retries permitted until 2026-09-17 13:15:57.349490283 +0000 UTC m=+66.254058938 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret") pod "global-pull-secret-syncer-p4p6z" (UID: "ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084") : object "kube-system"/"original-pull-secret" not registered Sep 17 13:15:25.352766 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.352727 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e063f7d-b387-4af5-944e-d77ffefceb18-metrics-certs\") pod \"network-metrics-daemon-lnvpk\" (UID: \"9e063f7d-b387-4af5-944e-d77ffefceb18\") " pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:25.512372 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.512338 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnvpk" Sep 17 13:15:25.603319 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.602674 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:25.603319 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.603162 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:25.605722 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.605699 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 13:15:25.605835 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.605706 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 13:15:25.605835 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.605710 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 13:15:25.606515 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.606495 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kgd8g\"" Sep 17 13:15:25.780987 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.780916 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9lbh4" event={"ID":"acaee4a3-6596-4bdf-83e3-438894602170","Type":"ContainerStarted","Data":"8a1ee98667893f4d0688f33eaab414f744fe704fbec45f017d1c90d07ecbadef"} Sep 17 13:15:25.783543 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.783449 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8f5x" event={"ID":"111aa84c-189b-489c-b075-226fee25b80d","Type":"ContainerStarted","Data":"f516c7a1f3fded6f55fb2663753fb6fc52b94616d3f30ea7bbf2db3bb696a6ec"} Sep 17 13:15:25.783543 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.783507 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8f5x" event={"ID":"111aa84c-189b-489c-b075-226fee25b80d","Type":"ContainerStarted","Data":"5a661c178abe0ecbacd6a77a2d708cc2d5cc8682b0fe905f6161c17527c1e5bb"} Sep 17 13:15:25.793456 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.793425 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9d4n" event={"ID":"644b480b-08f1-4fe9-b3db-177aaa8bcf46","Type":"ContainerStarted","Data":"395ec0f7f9f08ac7d6656b4fc7784c0a8d480114907529f594082ffb3446cc0d"} Sep 17 13:15:25.807540 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:25.807467 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lnvpk"] Sep 17 13:15:25.810707 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:25.810664 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e063f7d_b387_4af5_944e_d77ffefceb18.slice/crio-9f66bcde50bbb6746e44cfbf518c0731e182fd7b30d5213fc8cdc5b7ea06f1ad WatchSource:0}: Error finding container 9f66bcde50bbb6746e44cfbf518c0731e182fd7b30d5213fc8cdc5b7ea06f1ad: Status 404 returned error can't find the container with id 9f66bcde50bbb6746e44cfbf518c0731e182fd7b30d5213fc8cdc5b7ea06f1ad Sep 17 13:15:26.551112 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.551080 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68948f6644-rjbjn"] Sep 17 13:15:26.554442 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.554423 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.557165 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.557140 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kv6zw\"" Sep 17 13:15:26.557292 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.557165 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 17 13:15:26.557379 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.557369 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 17 13:15:26.557477 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.557457 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 17 13:15:26.557708 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.557690 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 17 13:15:26.557877 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.557736 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 17 13:15:26.558071 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.558041 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 17 13:15:26.558321 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.558234 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 17 13:15:26.563523 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.563475 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68948f6644-rjbjn"] Sep 17 13:15:26.660358 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.660327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-service-ca\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.660527 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.660370 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-config\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.660527 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.660437 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-oauth-config\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.660527 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.660458 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2sql\" (UniqueName: \"kubernetes.io/projected/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-kube-api-access-t2sql\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.660527 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.660497 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-oauth-serving-cert\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.660681 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.660538 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-serving-cert\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.762546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.761797 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-service-ca\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.762546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.761849 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-config\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.762546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.761910 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-oauth-config\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.762546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.761938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2sql\" (UniqueName: \"kubernetes.io/projected/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-kube-api-access-t2sql\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.762546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.761981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-oauth-serving-cert\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.762546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.762011 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-serving-cert\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.763460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.763429 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-oauth-serving-cert\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.763755 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.763471 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-service-ca\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.763755 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.763563 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-config\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.768783 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.768723 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-oauth-config\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.768901 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.768810 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-serving-cert\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.772443 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.771453 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2"] Sep 17 13:15:26.776608 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.776586 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2sql\" (UniqueName: \"kubernetes.io/projected/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-kube-api-access-t2sql\") pod \"console-68948f6644-rjbjn\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.778426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.777811 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:26.779638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.779613 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 17 13:15:26.779728 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.779642 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 17 13:15:26.779973 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.779951 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 13:15:26.780141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.780123 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 13:15:26.780203 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.780123 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-p89z6\"" Sep 17 13:15:26.780203 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.780167 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 13:15:26.789035 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.789016 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2"] Sep 17 13:15:26.798802 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.798771 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8f5x" event={"ID":"111aa84c-189b-489c-b075-226fee25b80d","Type":"ContainerStarted","Data":"2e5b9e0acb8e5ab9d6c3acc924006df5a35393c45246aab4b29332bbfa3d2483"} Sep 17 13:15:26.800153 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.800129 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnvpk" event={"ID":"9e063f7d-b387-4af5-944e-d77ffefceb18","Type":"ContainerStarted","Data":"9f66bcde50bbb6746e44cfbf518c0731e182fd7b30d5213fc8cdc5b7ea06f1ad"} Sep 17 13:15:26.865668 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.865593 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:26.963612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.963578 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ba07348-1f3b-4938-8e44-b2984fabce39-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:26.963784 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.963641 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ba07348-1f3b-4938-8e44-b2984fabce39-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:26.963784 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.963672 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bm8q\" (UniqueName: \"kubernetes.io/projected/4ba07348-1f3b-4938-8e44-b2984fabce39-kube-api-access-9bm8q\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:26.963784 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:26.963715 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ba07348-1f3b-4938-8e44-b2984fabce39-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.065510 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.064981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ba07348-1f3b-4938-8e44-b2984fabce39-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.065510 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.065041 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bm8q\" (UniqueName: \"kubernetes.io/projected/4ba07348-1f3b-4938-8e44-b2984fabce39-kube-api-access-9bm8q\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.065510 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.065080 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ba07348-1f3b-4938-8e44-b2984fabce39-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.065510 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.065138 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ba07348-1f3b-4938-8e44-b2984fabce39-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.065926 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.065883 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ba07348-1f3b-4938-8e44-b2984fabce39-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.068007 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.067978 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ba07348-1f3b-4938-8e44-b2984fabce39-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.068256 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.068233 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4ba07348-1f3b-4938-8e44-b2984fabce39-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.074118 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.074089 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bm8q\" (UniqueName: \"kubernetes.io/projected/4ba07348-1f3b-4938-8e44-b2984fabce39-kube-api-access-9bm8q\") pod \"prometheus-operator-68bcd7f987-vbhf2\" (UID: \"4ba07348-1f3b-4938-8e44-b2984fabce39\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:27.091842 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:27.091816 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" Sep 17 13:15:29.675689 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:29.675386 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2"] Sep 17 13:15:29.678061 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:29.678032 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68948f6644-rjbjn"] Sep 17 13:15:29.688177 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:29.688153 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a77ec8b_def7_43f9_bc3e_c2fad3c64b17.slice/crio-d71817b9dbc66bc32b05fcb4e7cdad65498de2ab8ffdbca004981c5473b19d93 WatchSource:0}: Error finding container d71817b9dbc66bc32b05fcb4e7cdad65498de2ab8ffdbca004981c5473b19d93: Status 404 returned error can't find the container with id d71817b9dbc66bc32b05fcb4e7cdad65498de2ab8ffdbca004981c5473b19d93 Sep 17 13:15:29.688465 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:29.688422 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba07348_1f3b_4938_8e44_b2984fabce39.slice/crio-64a69d03817afb273188955e6dbe01a497c910a745d604810787141273a33607 WatchSource:0}: Error finding container 64a69d03817afb273188955e6dbe01a497c910a745d604810787141273a33607: Status 404 returned error can't find the container with id 64a69d03817afb273188955e6dbe01a497c910a745d604810787141273a33607 Sep 17 13:15:29.810676 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:29.810585 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68948f6644-rjbjn" event={"ID":"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17","Type":"ContainerStarted","Data":"d71817b9dbc66bc32b05fcb4e7cdad65498de2ab8ffdbca004981c5473b19d93"} Sep 17 13:15:29.811895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:29.811867 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" event={"ID":"4ba07348-1f3b-4938-8e44-b2984fabce39","Type":"ContainerStarted","Data":"64a69d03817afb273188955e6dbe01a497c910a745d604810787141273a33607"} Sep 17 13:15:30.821081 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:30.821047 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8f5x" event={"ID":"111aa84c-189b-489c-b075-226fee25b80d","Type":"ContainerStarted","Data":"7bc421253b714ae8978153d244f6ee11c78f0701962f72d50ba1e7308ae223e4"} Sep 17 13:15:30.825912 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:30.825881 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnvpk" event={"ID":"9e063f7d-b387-4af5-944e-d77ffefceb18","Type":"ContainerStarted","Data":"6dae3de87fc726703422d5bc9f1bd306e5c9ac71115bf134f5b06bca88e0df0b"} Sep 17 13:15:30.832288 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:30.832247 2575 generic.go:356] "Generic (PLEG): container finished" podID="9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b" containerID="1047de9d4a11b75d0493d6d8c2e9a47ac225046a97cc87a572b643ce77191636" exitCode=0 Sep 17 13:15:30.832910 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:30.832889 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerDied","Data":"1047de9d4a11b75d0493d6d8c2e9a47ac225046a97cc87a572b643ce77191636"} Sep 17 13:15:30.836284 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:30.836220 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9d4n" event={"ID":"644b480b-08f1-4fe9-b3db-177aaa8bcf46","Type":"ContainerStarted","Data":"5cb2408681d8ceeb7e0751c2092a7c634994a538a0f5577d8be4297d85cc6e7d"} Sep 17 13:15:30.837955 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:30.837932 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9lbh4" event={"ID":"acaee4a3-6596-4bdf-83e3-438894602170","Type":"ContainerStarted","Data":"8d483adc5bbcdbf0649601930b740fe706925324e014f8d7bc8f8eec1e430950"} Sep 17 13:15:30.867919 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:30.867257 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g8f5x" podStartSLOduration=1.547816289 podStartE2EDuration="6.867236169s" totalImagesPullingTime="5.31941988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:15:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:30.844613406 +0000 UTC m=+39.749182110" watchObservedRunningTime="2026-09-17 13:15:30.867236169 +0000 UTC m=+39.771804847" Sep 17 13:15:30.883250 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:30.882603 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9lbh4" podStartSLOduration=2.557373913 podStartE2EDuration="6.882582525s" totalImagesPullingTime="4.325208612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:30.881581168 +0000 UTC m=+39.786149877" watchObservedRunningTime="2026-09-17 13:15:30.882582525 +0000 UTC m=+39.787151204" Sep 17 13:15:31.845211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:31.845060 2575 generic.go:356] "Generic (PLEG): container finished" podID="9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b" containerID="eb780fd0dd5fb9d475e218c17e10a879967c1a678a49bc4f5a834f67e14c41be" exitCode=0 Sep 17 13:15:31.845211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:31.845101 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerDied","Data":"eb780fd0dd5fb9d475e218c17e10a879967c1a678a49bc4f5a834f67e14c41be"} Sep 17 13:15:31.847432 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:31.847315 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9d4n" event={"ID":"644b480b-08f1-4fe9-b3db-177aaa8bcf46","Type":"ContainerStarted","Data":"6cb5506aca245900fb91d1b264b66d63ab9662ca010d9a18d330fbbbf3c606d3"} Sep 17 13:15:31.847432 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:31.847384 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:31.849477 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:31.849449 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnvpk" event={"ID":"9e063f7d-b387-4af5-944e-d77ffefceb18","Type":"ContainerStarted","Data":"1b4d0b759c94339f463a0536b53cf92d8d0306e0780c86bebd3676e3099c54a2"} Sep 17 13:15:31.885031 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:31.884977 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lnvpk" podStartSLOduration=35.215246704 podStartE2EDuration="39.884944553s" totalImagesPullingTime="4.669697849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:31.884026223 +0000 UTC m=+40.788594915" watchObservedRunningTime="2026-09-17 13:15:31.884944553 +0000 UTC m=+40.789513232" Sep 17 13:15:32.855767 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:32.855725 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" event={"ID":"4ba07348-1f3b-4938-8e44-b2984fabce39","Type":"ContainerStarted","Data":"65e779840d13378e9d782d6c99c2c818627c240949570e6f620a081da412d15b"} Sep 17 13:15:32.859829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:32.859798 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" event={"ID":"9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b","Type":"ContainerStarted","Data":"112b41b9ecd33413a9f9107c0bfd6726fbdf4147249f3bd322f071dae8b3983b"} Sep 17 13:15:32.881475 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:32.881421 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r9d4n" podStartSLOduration=4.491074018 podStartE2EDuration="8.88139895s" totalImagesPullingTime="4.390324932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:31.903611382 +0000 UTC m=+40.808180059" watchObservedRunningTime="2026-09-17 13:15:32.88139895 +0000 UTC m=+41.785967629" Sep 17 13:15:32.882212 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:32.881887 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pn7zx" podStartSLOduration=16.969906753 podStartE2EDuration="40.881873918s" totalImagesPullingTime="23.911967165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:32.880956182 +0000 UTC m=+41.785524883" watchObservedRunningTime="2026-09-17 13:15:32.881873918 +0000 UTC m=+41.786442595" Sep 17 13:15:33.864522 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:33.864428 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" event={"ID":"4ba07348-1f3b-4938-8e44-b2984fabce39","Type":"ContainerStarted","Data":"94a39dc7b0e2133ce026d949d48682d268cb962093a8607a52eaa2fa71e25311"} Sep 17 13:15:33.865945 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:33.865922 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68948f6644-rjbjn" event={"ID":"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17","Type":"ContainerStarted","Data":"b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b"} Sep 17 13:15:33.881408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:33.881369 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vbhf2" podStartSLOduration=5.70142749 podStartE2EDuration="7.881351091s" totalImagesPullingTime="2.179923601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:33.880601267 +0000 UTC m=+42.785169944" watchObservedRunningTime="2026-09-17 13:15:33.881351091 +0000 UTC m=+42.785919768" Sep 17 13:15:33.899199 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:33.899154 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68948f6644-rjbjn" podStartSLOduration=4.047002638 podStartE2EDuration="7.899134631s" totalImagesPullingTime="3.852131993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:33.898313197 +0000 UTC m=+42.802881890" watchObservedRunningTime="2026-09-17 13:15:33.899134631 +0000 UTC m=+42.803703307" Sep 17 13:15:36.128292 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.128242 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb"] Sep 17 13:15:36.164168 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.164143 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-8596r"] Sep 17 13:15:36.164328 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.164256 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.166780 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.166759 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 17 13:15:36.167380 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.167350 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zxdcq\"" Sep 17 13:15:36.167519 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.167350 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 17 13:15:36.180148 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.180127 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb"] Sep 17 13:15:36.180259 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.180167 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-44nw6"] Sep 17 13:15:36.180259 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.180233 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.182295 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.182255 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 17 13:15:36.182519 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.182500 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 17 13:15:36.182607 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.182557 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cclq8\"" Sep 17 13:15:36.182607 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.182589 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 17 13:15:36.205251 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.205220 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-8596r"] Sep 17 13:15:36.205581 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.205502 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.208764 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.208738 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 13:15:36.208870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.208788 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b5nv6\"" Sep 17 13:15:36.208870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.208861 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 13:15:36.208980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.208961 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 13:15:36.234670 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234644 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-sys\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.234774 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234676 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-tls\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.234774 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234695 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.234774 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234725 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqt4v\" (UniqueName: \"kubernetes.io/projected/d4e0e957-6576-4a42-acac-75de9179551e-kube-api-access-jqt4v\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.234873 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234793 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.234873 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234808 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvb9x\" (UniqueName: \"kubernetes.io/projected/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-kube-api-access-gvb9x\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.234873 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234845 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-accelerators-collector-config\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.234965 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234869 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4e0e957-6576-4a42-acac-75de9179551e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.234965 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1d2c119-7d1d-4520-8e87-b05f083d3a12-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.234965 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234910 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.234965 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234951 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.235080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.234981 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1d2c119-7d1d-4520-8e87-b05f083d3a12-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.235080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.235015 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-root\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.235080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.235035 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4e0e957-6576-4a42-acac-75de9179551e-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.235080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.235067 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-textfile\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.235192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.235092 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-metrics-client-ca\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.235192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.235119 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-wtmp\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.235192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.235145 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4e0e957-6576-4a42-acac-75de9179551e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.235192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.235174 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmv7p\" (UniqueName: \"kubernetes.io/projected/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-api-access-wmv7p\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.336460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336424 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-tls\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.336460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336464 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336485 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqt4v\" (UniqueName: \"kubernetes.io/projected/d4e0e957-6576-4a42-acac-75de9179551e-kube-api-access-jqt4v\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336517 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336533 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvb9x\" (UniqueName: \"kubernetes.io/projected/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-kube-api-access-gvb9x\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336550 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-accelerators-collector-config\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336564 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4e0e957-6576-4a42-acac-75de9179551e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336580 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1d2c119-7d1d-4520-8e87-b05f083d3a12-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336605 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336630 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336655 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1d2c119-7d1d-4520-8e87-b05f083d3a12-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336683 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-root\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.336715 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336699 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4e0e957-6576-4a42-acac-75de9179551e-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336725 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-textfile\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336770 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-metrics-client-ca\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-wtmp\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4e0e957-6576-4a42-acac-75de9179551e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.336999 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c1d2c119-7d1d-4520-8e87-b05f083d3a12-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.337029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmv7p\" (UniqueName: \"kubernetes.io/projected/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-api-access-wmv7p\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.337082 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-root\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.337090 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-sys\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.337243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.337195 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-sys\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.337700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.337410 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.337700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.337649 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1d2c119-7d1d-4520-8e87-b05f083d3a12-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.337843 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.337817 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4e0e957-6576-4a42-acac-75de9179551e-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.337909 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.337789 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-wtmp\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.338421 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.338394 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-textfile\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.338867 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.338840 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-accelerators-collector-config\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.338955 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.338918 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-metrics-client-ca\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.340265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.340242 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.340485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.340464 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.340573 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.340480 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-tls\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.340573 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.340464 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d4e0e957-6576-4a42-acac-75de9179551e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.340573 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.340482 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4e0e957-6576-4a42-acac-75de9179551e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.340735 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.340709 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.345729 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.345703 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqt4v\" (UniqueName: \"kubernetes.io/projected/d4e0e957-6576-4a42-acac-75de9179551e-kube-api-access-jqt4v\") pod \"openshift-state-metrics-67dcf8c979-sdswb\" (UID: \"d4e0e957-6576-4a42-acac-75de9179551e\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.346014 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.345998 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvb9x\" (UniqueName: \"kubernetes.io/projected/0cecb85f-7a0c-477f-8e92-34a77ad52f4e-kube-api-access-gvb9x\") pod \"node-exporter-44nw6\" (UID: \"0cecb85f-7a0c-477f-8e92-34a77ad52f4e\") " pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.346118 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.346100 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmv7p\" (UniqueName: \"kubernetes.io/projected/c1d2c119-7d1d-4520-8e87-b05f083d3a12-kube-api-access-wmv7p\") pod \"kube-state-metrics-59fbdd77d-8596r\" (UID: \"c1d2c119-7d1d-4520-8e87-b05f083d3a12\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.473702 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.473624 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" Sep 17 13:15:36.489781 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.489762 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" Sep 17 13:15:36.515184 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.515164 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-44nw6" Sep 17 13:15:36.552783 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:36.552744 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cecb85f_7a0c_477f_8e92_34a77ad52f4e.slice/crio-584c41817c5aacaf361ca2aaf52fada13f41e9f062ecf541a40b5f92312fa23a WatchSource:0}: Error finding container 584c41817c5aacaf361ca2aaf52fada13f41e9f062ecf541a40b5f92312fa23a: Status 404 returned error can't find the container with id 584c41817c5aacaf361ca2aaf52fada13f41e9f062ecf541a40b5f92312fa23a Sep 17 13:15:36.773917 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.773887 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb"] Sep 17 13:15:36.776384 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:36.776354 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e0e957_6576_4a42_acac_75de9179551e.slice/crio-a9dafedbc8921067a5a3dda1089b92931dc34738dfccab6aba360d8f78caf1fe WatchSource:0}: Error finding container a9dafedbc8921067a5a3dda1089b92931dc34738dfccab6aba360d8f78caf1fe: Status 404 returned error can't find the container with id a9dafedbc8921067a5a3dda1089b92931dc34738dfccab6aba360d8f78caf1fe Sep 17 13:15:36.811688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.811580 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-8596r"] Sep 17 13:15:36.814030 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:36.814009 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d2c119_7d1d_4520_8e87_b05f083d3a12.slice/crio-265ae3fc4aa92322b57a5625ef34801d1dc73d2d71c15d761c88826bc2f4ecac WatchSource:0}: Error finding container 265ae3fc4aa92322b57a5625ef34801d1dc73d2d71c15d761c88826bc2f4ecac: Status 404 returned error can't find the container with id 265ae3fc4aa92322b57a5625ef34801d1dc73d2d71c15d761c88826bc2f4ecac Sep 17 13:15:36.865988 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.865966 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:36.866114 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.866042 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:36.871389 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.871365 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:36.877666 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.877634 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44nw6" event={"ID":"0cecb85f-7a0c-477f-8e92-34a77ad52f4e","Type":"ContainerStarted","Data":"584c41817c5aacaf361ca2aaf52fada13f41e9f062ecf541a40b5f92312fa23a"} Sep 17 13:15:36.878843 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.878816 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" event={"ID":"c1d2c119-7d1d-4520-8e87-b05f083d3a12","Type":"ContainerStarted","Data":"265ae3fc4aa92322b57a5625ef34801d1dc73d2d71c15d761c88826bc2f4ecac"} Sep 17 13:15:36.880535 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.880515 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" event={"ID":"d4e0e957-6576-4a42-acac-75de9179551e","Type":"ContainerStarted","Data":"a9dafedbc8921067a5a3dda1089b92931dc34738dfccab6aba360d8f78caf1fe"} Sep 17 13:15:36.884200 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:36.884179 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:15:37.255006 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.254922 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:15:37.273361 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.272202 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.274417 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.274266 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 17 13:15:37.275201 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.275181 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 17 13:15:37.275444 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.275427 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 17 13:15:37.276562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.275657 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 17 13:15:37.276562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.275800 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 17 13:15:37.276562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.275840 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 17 13:15:37.276562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.275844 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 17 13:15:37.276562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.275904 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 17 13:15:37.276562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.276207 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 17 13:15:37.276562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.276413 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9d445\"" Sep 17 13:15:37.276886 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.276637 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:15:37.345751 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345721 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.345913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345758 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.345913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345786 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-config-out\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.345913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345818 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.345913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.345913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345863 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-web-config\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.345913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345877 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.346177 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345927 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.346177 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.346177 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.345981 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-config-volume\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.346177 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.346003 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r52dn\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-kube-api-access-r52dn\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.346177 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.346026 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.346177 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.346109 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447108 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447066 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447108 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447107 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447357 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447125 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-config-out\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447357 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447310 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447357 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447347 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447386 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-web-config\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447408 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447439 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447464 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447490 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-config-volume\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447522 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r52dn\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-kube-api-access-r52dn\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447527 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447562 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.447636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.447595 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.448022 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:15:37.447665 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle podName:68420cdc-3117-4eb5-a71e-da01d54244ad nodeName:}" failed. No retries permitted until 2026-09-17 13:15:37.947642519 +0000 UTC m=+46.852211195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad") : configmap references non-existent config key: ca-bundle.crt Sep 17 13:15:37.448788 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.448207 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.450787 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.450457 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.450787 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.450507 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.452838 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.452817 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-config-volume\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.453807 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.453784 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.453807 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.453803 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.453990 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.453975 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-config-out\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.454343 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.454322 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.455581 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.455505 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-web-config\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.455581 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.455554 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.458113 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.458085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r52dn\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-kube-api-access-r52dn\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.629574 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.629385 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c4dd4b857-98m6b"] Sep 17 13:15:37.648459 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.648413 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c4dd4b857-98m6b"] Sep 17 13:15:37.648612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.648594 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.656895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.656871 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 17 13:15:37.750801 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.750772 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-oauth-config\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.750801 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.750803 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-trusted-ca-bundle\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.751017 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.750822 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-service-ca\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.751017 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.750863 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-console-config\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.751017 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.750910 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-oauth-serving-cert\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.751017 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.750991 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2khx\" (UniqueName: \"kubernetes.io/projected/467a959a-d853-4d6a-b63c-d0b1b5348848-kube-api-access-k2khx\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.751017 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.751017 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-serving-cert\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.851948 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.851920 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2khx\" (UniqueName: \"kubernetes.io/projected/467a959a-d853-4d6a-b63c-d0b1b5348848-kube-api-access-k2khx\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.852148 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.851966 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-serving-cert\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.852148 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.852012 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-oauth-config\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.852148 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.852032 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-trusted-ca-bundle\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.852148 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.852058 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-service-ca\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.852148 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.852106 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-console-config\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.852148 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.852129 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-oauth-serving-cert\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.852780 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.852757 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-oauth-serving-cert\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.853127 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.853094 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-trusted-ca-bundle\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.853406 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.853330 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-service-ca\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.853730 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.853707 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-console-config\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.855600 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.855579 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-serving-cert\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.855747 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.855725 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-oauth-config\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.859957 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.859936 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2khx\" (UniqueName: \"kubernetes.io/projected/467a959a-d853-4d6a-b63c-d0b1b5348848-kube-api-access-k2khx\") pod \"console-c4dd4b857-98m6b\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:37.885862 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.885803 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" event={"ID":"d4e0e957-6576-4a42-acac-75de9179551e","Type":"ContainerStarted","Data":"b4521bff384dd3d750ccd0575250726d152fd28d8649acc74dafba75dfc5c2b6"} Sep 17 13:15:37.885950 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.885866 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" event={"ID":"d4e0e957-6576-4a42-acac-75de9179551e","Type":"ContainerStarted","Data":"372bff1277bff462f2bcb386c791ed18d55ac265fcf8f39654b1c7a2d909d816"} Sep 17 13:15:37.953196 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.953155 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.954381 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.954357 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:37.962122 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:37.962103 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:38.185539 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:38.185458 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:15:39.137255 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.137226 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d77f96749-ctbf6"] Sep 17 13:15:39.168365 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.167961 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d77f96749-ctbf6"] Sep 17 13:15:39.168365 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.168079 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.172638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.170841 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 17 13:15:39.172638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.171165 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 17 13:15:39.172638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.171395 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 17 13:15:39.172638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.171817 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9f8ld\"" Sep 17 13:15:39.172638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.172002 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 17 13:15:39.172638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.172193 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 17 13:15:39.172638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.172404 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6dmcunsuotcd9\"" Sep 17 13:15:39.264872 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.264850 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89cd6a39-2f1f-40bf-89f0-7aec56692c76-metrics-client-ca\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.264983 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.264915 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.264983 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.264960 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-tls\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.265087 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.264987 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.265139 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.265086 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khj4q\" (UniqueName: \"kubernetes.io/projected/89cd6a39-2f1f-40bf-89f0-7aec56692c76-kube-api-access-khj4q\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.265192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.265135 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.265192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.265163 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-grpc-tls\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.265295 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.265201 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.366370 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.366336 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.366493 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.366379 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89cd6a39-2f1f-40bf-89f0-7aec56692c76-metrics-client-ca\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.366493 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.366418 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.366824 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.366455 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-tls\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.366824 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.366793 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.366961 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.366873 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khj4q\" (UniqueName: \"kubernetes.io/projected/89cd6a39-2f1f-40bf-89f0-7aec56692c76-kube-api-access-khj4q\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.366961 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.366929 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.367032 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.366964 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-grpc-tls\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.368340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.368316 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89cd6a39-2f1f-40bf-89f0-7aec56692c76-metrics-client-ca\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.377982 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.377958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.378164 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.378005 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-tls\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.378463 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.378436 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-grpc-tls\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.381863 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.380120 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khj4q\" (UniqueName: \"kubernetes.io/projected/89cd6a39-2f1f-40bf-89f0-7aec56692c76-kube-api-access-khj4q\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.381863 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.380518 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.382864 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.382841 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.382944 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.382840 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/89cd6a39-2f1f-40bf-89f0-7aec56692c76-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d77f96749-ctbf6\" (UID: \"89cd6a39-2f1f-40bf-89f0-7aec56692c76\") " pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.398900 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.398855 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c4dd4b857-98m6b"] Sep 17 13:15:39.402473 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:39.402440 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467a959a_d853_4d6a_b63c_d0b1b5348848.slice/crio-31c5b8e9fec17e565da5d673ce3d54dd4d1a9f72473d10c87253403fe373a21b WatchSource:0}: Error finding container 31c5b8e9fec17e565da5d673ce3d54dd4d1a9f72473d10c87253403fe373a21b: Status 404 returned error can't find the container with id 31c5b8e9fec17e565da5d673ce3d54dd4d1a9f72473d10c87253403fe373a21b Sep 17 13:15:39.409428 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.409409 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:15:39.414078 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:39.414040 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68420cdc_3117_4eb5_a71e_da01d54244ad.slice/crio-7535748066d43438eb1e14ef7b24e1286c69cafb88680323825f4b1b0a4fc391 WatchSource:0}: Error finding container 7535748066d43438eb1e14ef7b24e1286c69cafb88680323825f4b1b0a4fc391: Status 404 returned error can't find the container with id 7535748066d43438eb1e14ef7b24e1286c69cafb88680323825f4b1b0a4fc391 Sep 17 13:15:39.504131 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.504106 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:39.798760 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.798609 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d77f96749-ctbf6"] Sep 17 13:15:39.801093 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:39.801071 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89cd6a39_2f1f_40bf_89f0_7aec56692c76.slice/crio-225f080d64048fbb77810c95f41803a8c697275b115ad3d0f7ceac34aca7e5b9 WatchSource:0}: Error finding container 225f080d64048fbb77810c95f41803a8c697275b115ad3d0f7ceac34aca7e5b9: Status 404 returned error can't find the container with id 225f080d64048fbb77810c95f41803a8c697275b115ad3d0f7ceac34aca7e5b9 Sep 17 13:15:39.894337 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.894246 2575 generic.go:356] "Generic (PLEG): container finished" podID="0cecb85f-7a0c-477f-8e92-34a77ad52f4e" containerID="03786a7eff049eee854886e587057859bb7c3bed052519c905f2433cd34dcff3" exitCode=0 Sep 17 13:15:39.894462 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.894338 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44nw6" event={"ID":"0cecb85f-7a0c-477f-8e92-34a77ad52f4e","Type":"ContainerDied","Data":"03786a7eff049eee854886e587057859bb7c3bed052519c905f2433cd34dcff3"} Sep 17 13:15:39.896584 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.896561 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" event={"ID":"c1d2c119-7d1d-4520-8e87-b05f083d3a12","Type":"ContainerStarted","Data":"4c014c0a2bfd18062efe369c5cc83b6bf526f028f82ee6064894947884831718"} Sep 17 13:15:39.896703 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.896613 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" event={"ID":"c1d2c119-7d1d-4520-8e87-b05f083d3a12","Type":"ContainerStarted","Data":"954584f13c929fdbf35f5453c8914cdba5cc9ee1563b0a24dcaa902415576bb5"} Sep 17 13:15:39.896703 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.896642 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" event={"ID":"c1d2c119-7d1d-4520-8e87-b05f083d3a12","Type":"ContainerStarted","Data":"33216b0f9c382f39bfdf1aa09274f197dffac0a86d0823d50a7cda45c99a9d69"} Sep 17 13:15:39.897815 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.897795 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerStarted","Data":"7535748066d43438eb1e14ef7b24e1286c69cafb88680323825f4b1b0a4fc391"} Sep 17 13:15:39.899215 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.899189 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4dd4b857-98m6b" event={"ID":"467a959a-d853-4d6a-b63c-d0b1b5348848","Type":"ContainerStarted","Data":"7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4"} Sep 17 13:15:39.899318 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.899226 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4dd4b857-98m6b" event={"ID":"467a959a-d853-4d6a-b63c-d0b1b5348848","Type":"ContainerStarted","Data":"31c5b8e9fec17e565da5d673ce3d54dd4d1a9f72473d10c87253403fe373a21b"} Sep 17 13:15:39.901446 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.901423 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" event={"ID":"d4e0e957-6576-4a42-acac-75de9179551e","Type":"ContainerStarted","Data":"c76c33d1a478e6bb1c82b7508f5280aaf8233bf0d0f070a1ecea5f0ca8ffda6d"} Sep 17 13:15:39.902549 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.902532 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" event={"ID":"89cd6a39-2f1f-40bf-89f0-7aec56692c76","Type":"ContainerStarted","Data":"225f080d64048fbb77810c95f41803a8c697275b115ad3d0f7ceac34aca7e5b9"} Sep 17 13:15:39.938027 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.937913 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-sdswb" podStartSLOduration=1.8403226529999999 podStartE2EDuration="3.937898365s" totalImagesPullingTime="2.097575712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:39.936402737 +0000 UTC m=+48.840971412" watchObservedRunningTime="2026-09-17 13:15:39.937898365 +0000 UTC m=+48.842467040" Sep 17 13:15:39.952342 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.952258 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-8596r" podStartSLOduration=1.650619393 podStartE2EDuration="3.952238368s" totalImagesPullingTime="2.301618975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:39.951824503 +0000 UTC m=+48.856393207" watchObservedRunningTime="2026-09-17 13:15:39.952238368 +0000 UTC m=+48.856807046" Sep 17 13:15:39.968814 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:39.968778 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c4dd4b857-98m6b" podStartSLOduration=2.968766447 podStartE2EDuration="2.968766447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:39.967266348 +0000 UTC m=+48.871835055" watchObservedRunningTime="2026-09-17 13:15:39.968766447 +0000 UTC m=+48.873335124" Sep 17 13:15:40.471054 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.470963 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6565d9b67d-kcl5n"] Sep 17 13:15:40.482760 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.481806 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.482760 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.482653 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6565d9b67d-kcl5n"] Sep 17 13:15:40.486597 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.484943 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8re2u2kegmh7f\"" Sep 17 13:15:40.486597 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.485226 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 17 13:15:40.486597 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.485239 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nv7ls\"" Sep 17 13:15:40.486597 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.485359 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 17 13:15:40.486597 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.485522 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 17 13:15:40.486597 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.485567 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 17 13:15:40.579661 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.579622 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0004302a-59e3-4460-b964-d4b225a7e548-audit-log\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.579839 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.579695 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0004302a-59e3-4460-b964-d4b225a7e548-metrics-server-audit-profiles\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.579839 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.579724 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-secret-metrics-server-tls\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.579839 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.579755 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqm86\" (UniqueName: \"kubernetes.io/projected/0004302a-59e3-4460-b964-d4b225a7e548-kube-api-access-sqm86\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.579998 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.579841 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0004302a-59e3-4460-b964-d4b225a7e548-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.579998 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.579882 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-client-ca-bundle\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.579998 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.579912 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-secret-metrics-server-client-certs\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.680458 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.680419 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqm86\" (UniqueName: \"kubernetes.io/projected/0004302a-59e3-4460-b964-d4b225a7e548-kube-api-access-sqm86\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.680631 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.680505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0004302a-59e3-4460-b964-d4b225a7e548-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.680631 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.680537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-client-ca-bundle\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.680631 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.680568 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-secret-metrics-server-client-certs\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.680851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.680639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0004302a-59e3-4460-b964-d4b225a7e548-audit-log\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.680851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.680679 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0004302a-59e3-4460-b964-d4b225a7e548-metrics-server-audit-profiles\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.680851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.680716 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-secret-metrics-server-tls\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.681550 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.681238 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0004302a-59e3-4460-b964-d4b225a7e548-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.681665 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.681574 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0004302a-59e3-4460-b964-d4b225a7e548-audit-log\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.684299 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.684110 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-secret-metrics-server-tls\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.684299 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.684240 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-client-ca-bundle\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.685010 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.684975 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0004302a-59e3-4460-b964-d4b225a7e548-secret-metrics-server-client-certs\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.689220 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.689196 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqm86\" (UniqueName: \"kubernetes.io/projected/0004302a-59e3-4460-b964-d4b225a7e548-kube-api-access-sqm86\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.692180 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.691304 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0004302a-59e3-4460-b964-d4b225a7e548-metrics-server-audit-profiles\") pod \"metrics-server-6565d9b67d-kcl5n\" (UID: \"0004302a-59e3-4460-b964-d4b225a7e548\") " pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.797552 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.797522 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:15:40.908363 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.908331 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8"] Sep 17 13:15:40.921898 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.921867 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44nw6" event={"ID":"0cecb85f-7a0c-477f-8e92-34a77ad52f4e","Type":"ContainerStarted","Data":"5cf20707f17c669c9dd0dfc123bacf465b98ec978d7e7ffcc0eceacfb41d73e9"} Sep 17 13:15:40.922053 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.921908 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44nw6" event={"ID":"0cecb85f-7a0c-477f-8e92-34a77ad52f4e","Type":"ContainerStarted","Data":"24d29ad661c2a13ee1888cba0f5604538bba81df563ba9a5e30220c0b056854c"} Sep 17 13:15:40.922053 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.921991 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:40.924353 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.924331 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 17 13:15:40.924475 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.924365 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-thrqj\"" Sep 17 13:15:40.924558 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.924513 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8"] Sep 17 13:15:40.927563 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.927524 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-44nw6" podStartSLOduration=2.339404187 podStartE2EDuration="4.927507491s" totalImagesPullingTime="2.496636989s" totalInitContainerRuntime="91.466315ms" isStatefulPod=true podCreationTimestamp="2026-09-17 13:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:40.927128957 +0000 UTC m=+49.831697652" watchObservedRunningTime="2026-09-17 13:15:40.927507491 +0000 UTC m=+49.832076168" Sep 17 13:15:40.985488 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.985445 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnnd7\" (UniqueName: \"kubernetes.io/projected/bc97ce9d-1c8b-46bb-842f-36120db3ade5-kube-api-access-lnnd7\") pod \"monitoring-plugin-5d85778b9d-6n9j8\" (UID: \"bc97ce9d-1c8b-46bb-842f-36120db3ade5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:40.985693 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:40.985569 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc97ce9d-1c8b-46bb-842f-36120db3ade5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-6n9j8\" (UID: \"bc97ce9d-1c8b-46bb-842f-36120db3ade5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:41.087827 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.087749 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnnd7\" (UniqueName: \"kubernetes.io/projected/bc97ce9d-1c8b-46bb-842f-36120db3ade5-kube-api-access-lnnd7\") pod \"monitoring-plugin-5d85778b9d-6n9j8\" (UID: \"bc97ce9d-1c8b-46bb-842f-36120db3ade5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:41.087827 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.087798 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc97ce9d-1c8b-46bb-842f-36120db3ade5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-6n9j8\" (UID: \"bc97ce9d-1c8b-46bb-842f-36120db3ade5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:41.090294 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.090208 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc97ce9d-1c8b-46bb-842f-36120db3ade5-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-6n9j8\" (UID: \"bc97ce9d-1c8b-46bb-842f-36120db3ade5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:41.095682 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.095655 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnnd7\" (UniqueName: \"kubernetes.io/projected/bc97ce9d-1c8b-46bb-842f-36120db3ade5-kube-api-access-lnnd7\") pod \"monitoring-plugin-5d85778b9d-6n9j8\" (UID: \"bc97ce9d-1c8b-46bb-842f-36120db3ade5\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:41.244528 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.244492 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:41.407367 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.405948 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-699c65d854-48j7l"] Sep 17 13:15:41.422066 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.422038 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.423174 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.423148 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-699c65d854-48j7l"] Sep 17 13:15:41.424408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.424192 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 17 13:15:41.424408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.424237 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-j4xp7\"" Sep 17 13:15:41.424738 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.424718 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 17 13:15:41.424837 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.424754 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 17 13:15:41.424837 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.424792 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 17 13:15:41.424932 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.424920 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 17 13:15:41.430678 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.430658 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 17 13:15:41.490910 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.490876 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-telemeter-client-tls\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.491307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.490922 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-metrics-client-ca\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.491307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.490958 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-secret-telemeter-client\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.491307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.491035 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9pj6\" (UniqueName: \"kubernetes.io/projected/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-kube-api-access-f9pj6\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.491307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.491060 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.491307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.491084 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-serving-certs-ca-bundle\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.491307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.491109 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-federate-client-tls\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.491307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.491147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.592540 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.592505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.592725 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.592561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-serving-certs-ca-bundle\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.592725 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.592592 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-federate-client-tls\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.592725 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.592635 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.592725 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.592707 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-telemeter-client-tls\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.592986 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.592738 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-metrics-client-ca\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.592986 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.592769 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-secret-telemeter-client\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.592986 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.592808 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9pj6\" (UniqueName: \"kubernetes.io/projected/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-kube-api-access-f9pj6\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.593982 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.593957 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.594552 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.594527 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-serving-certs-ca-bundle\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.594916 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.594893 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-metrics-client-ca\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.597201 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.597170 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-telemeter-client-tls\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.597408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.597388 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.597484 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.597433 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-federate-client-tls\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.597596 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.597581 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-secret-telemeter-client\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.602069 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.602048 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9pj6\" (UniqueName: \"kubernetes.io/projected/f7bd1d70-d8d9-448a-a3b8-f25e23f887f5-kube-api-access-f9pj6\") pod \"telemeter-client-699c65d854-48j7l\" (UID: \"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5\") " pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.735330 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.735227 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" Sep 17 13:15:41.862415 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:41.862382 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r9d4n" Sep 17 13:15:42.289695 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.289537 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6565d9b67d-kcl5n"] Sep 17 13:15:42.298488 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:42.298450 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0004302a_59e3_4460_b964_d4b225a7e548.slice/crio-6c7626f90da7c1dbc8355a511a528d37861513cd43cd1f41691191bd34a055d8 WatchSource:0}: Error finding container 6c7626f90da7c1dbc8355a511a528d37861513cd43cd1f41691191bd34a055d8: Status 404 returned error can't find the container with id 6c7626f90da7c1dbc8355a511a528d37861513cd43cd1f41691191bd34a055d8 Sep 17 13:15:42.302115 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.302074 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8"] Sep 17 13:15:42.306558 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:42.306527 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc97ce9d_1c8b_46bb_842f_36120db3ade5.slice/crio-792f6159af9f4d6f20fdb8f737a56b9c3102ae98acdb3d6f04dcbf3707edd2da WatchSource:0}: Error finding container 792f6159af9f4d6f20fdb8f737a56b9c3102ae98acdb3d6f04dcbf3707edd2da: Status 404 returned error can't find the container with id 792f6159af9f4d6f20fdb8f737a56b9c3102ae98acdb3d6f04dcbf3707edd2da Sep 17 13:15:42.311508 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.311486 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-699c65d854-48j7l"] Sep 17 13:15:42.919934 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.919894 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" event={"ID":"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5","Type":"ContainerStarted","Data":"5386f7a266a7e480bf14d62f7f1fcee78e466885c5463772d5302f06fab04166"} Sep 17 13:15:42.921547 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.921515 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" event={"ID":"bc97ce9d-1c8b-46bb-842f-36120db3ade5","Type":"ContainerStarted","Data":"792f6159af9f4d6f20fdb8f737a56b9c3102ae98acdb3d6f04dcbf3707edd2da"} Sep 17 13:15:42.924136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.924106 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" event={"ID":"89cd6a39-2f1f-40bf-89f0-7aec56692c76","Type":"ContainerStarted","Data":"866e399fa01a148dcb65b47ed9f8f8804f84fe59162d0b6311bb0ede0b002d39"} Sep 17 13:15:42.924267 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.924182 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" event={"ID":"89cd6a39-2f1f-40bf-89f0-7aec56692c76","Type":"ContainerStarted","Data":"6e611c1e95248750c2303245d6970c1a5d5805d348638b6427f9e2817d3b5407"} Sep 17 13:15:42.924267 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.924242 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" event={"ID":"89cd6a39-2f1f-40bf-89f0-7aec56692c76","Type":"ContainerStarted","Data":"2192de665f0c166a2407623cb126a698409497c93e900deee980575376e5fb69"} Sep 17 13:15:42.925591 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.925553 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" event={"ID":"0004302a-59e3-4460-b964-d4b225a7e548","Type":"ContainerStarted","Data":"6c7626f90da7c1dbc8355a511a528d37861513cd43cd1f41691191bd34a055d8"} Sep 17 13:15:42.927447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.927414 2575 generic.go:356] "Generic (PLEG): container finished" podID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerID="fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f" exitCode=0 Sep 17 13:15:42.927554 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:42.927460 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerDied","Data":"fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f"} Sep 17 13:15:45.949467 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.949432 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerStarted","Data":"d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3"} Sep 17 13:15:45.951253 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.951225 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" event={"ID":"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5","Type":"ContainerStarted","Data":"88d80075dc53471e504e77c3da213ed2ae004334a964ae3a87d298e9f80217f2"} Sep 17 13:15:45.953501 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.953478 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" event={"ID":"bc97ce9d-1c8b-46bb-842f-36120db3ade5","Type":"ContainerStarted","Data":"8835300e7234c5ee66ddd5e15058392ae7ef9a50888da44af57b015776ecf81a"} Sep 17 13:15:45.955386 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.955332 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:45.969290 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.966237 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" Sep 17 13:15:45.969290 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.966455 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" event={"ID":"89cd6a39-2f1f-40bf-89f0-7aec56692c76","Type":"ContainerStarted","Data":"883471e8fabe9c748a5c711d730f7267d049f6a2e76949ce67c998ce60530e4c"} Sep 17 13:15:45.969445 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.969384 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" event={"ID":"0004302a-59e3-4460-b964-d4b225a7e548","Type":"ContainerStarted","Data":"9b1ea0dc030e0001a9e563b3648dd3f91ef833c20b9aad2a893ec2062cdb06de"} Sep 17 13:15:45.978299 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.978222 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-6n9j8" podStartSLOduration=2.490607519 podStartE2EDuration="5.9782042s" totalImagesPullingTime="3.487596681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:45.976454704 +0000 UTC m=+54.881023382" watchObservedRunningTime="2026-09-17 13:15:45.9782042 +0000 UTC m=+54.882772892" Sep 17 13:15:45.999837 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:45.999781 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" podStartSLOduration=2.504312585 podStartE2EDuration="5.999742091s" totalImagesPullingTime="3.495429506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:45.998041148 +0000 UTC m=+54.902609828" watchObservedRunningTime="2026-09-17 13:15:45.999742091 +0000 UTC m=+54.904310768" Sep 17 13:15:46.437878 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.437776 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c4dd4b857-98m6b"] Sep 17 13:15:46.460755 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.460730 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b7cd4689d-xqwkw"] Sep 17 13:15:46.462852 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.462830 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.473732 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.473711 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b7cd4689d-xqwkw"] Sep 17 13:15:46.540813 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.540773 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-oauth-config\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.540940 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.540828 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-oauth-serving-cert\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.540940 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.540858 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dphtm\" (UniqueName: \"kubernetes.io/projected/c23b0a37-fe33-443a-93e1-d4a22a561e27-kube-api-access-dphtm\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.540940 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.540901 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-config\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.540940 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.540918 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-service-ca\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.541097 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.540941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-trusted-ca-bundle\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.541097 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.541041 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-serving-cert\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.641362 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.641334 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-serving-cert\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.641575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.641380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-oauth-config\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.641575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.641395 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-oauth-serving-cert\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.641575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.641409 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dphtm\" (UniqueName: \"kubernetes.io/projected/c23b0a37-fe33-443a-93e1-d4a22a561e27-kube-api-access-dphtm\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.641575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.641429 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-config\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.641575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.641444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-service-ca\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.641575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.641472 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-trusted-ca-bundle\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.642157 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.642133 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-service-ca\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.642265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.642133 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-oauth-serving-cert\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.642265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.642225 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-config\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.642368 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.642344 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-trusted-ca-bundle\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.643934 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.643915 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-serving-cert\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.643998 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.643941 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-oauth-config\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.651765 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.651744 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dphtm\" (UniqueName: \"kubernetes.io/projected/c23b0a37-fe33-443a-93e1-d4a22a561e27-kube-api-access-dphtm\") pod \"console-5b7cd4689d-xqwkw\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.771637 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.771615 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:46.976717 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.976684 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerStarted","Data":"58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2"} Sep 17 13:15:46.977136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.976788 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerStarted","Data":"99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef"} Sep 17 13:15:46.977136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.976819 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerStarted","Data":"d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b"} Sep 17 13:15:46.977136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.976893 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerStarted","Data":"20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9"} Sep 17 13:15:46.977136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.976929 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerStarted","Data":"063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17"} Sep 17 13:15:46.978940 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.978913 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" event={"ID":"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5","Type":"ContainerStarted","Data":"3680cf9a2ff64ad0e1ca1628d1e7eb3c49f32b8517c16a681625e99d30b07699"} Sep 17 13:15:46.979068 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.979015 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" event={"ID":"f7bd1d70-d8d9-448a-a3b8-f25e23f887f5","Type":"ContainerStarted","Data":"5a2249c6ec2ee480eb041424bab15c85c4c21e493c76d4cb62a34fda883a3e06"} Sep 17 13:15:46.981948 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.981920 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" event={"ID":"89cd6a39-2f1f-40bf-89f0-7aec56692c76","Type":"ContainerStarted","Data":"91415f1cdb0437b835bbcac7cbb0f44deb669d18673bba9ccea20f5251f2a9f9"} Sep 17 13:15:46.982070 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:46.981976 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" event={"ID":"89cd6a39-2f1f-40bf-89f0-7aec56692c76","Type":"ContainerStarted","Data":"d182599c849feed940c452d5c64cbe6cb1ae956e328f9c87d63b76e050ad5a34"} Sep 17 13:15:47.005439 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.005401 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.472831428 podStartE2EDuration="10.005388566s" totalImagesPullingTime="5.501437094s" totalInitContainerRuntime="31.120044ms" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:47.00344437 +0000 UTC m=+55.908013057" watchObservedRunningTime="2026-09-17 13:15:47.005388566 +0000 UTC m=+55.909957243" Sep 17 13:15:47.028182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.028101 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" podStartSLOduration=2.528168684 podStartE2EDuration="8.028086777s" totalImagesPullingTime="5.499918093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:47.027225387 +0000 UTC m=+55.931794091" watchObservedRunningTime="2026-09-17 13:15:47.028086777 +0000 UTC m=+55.932655453" Sep 17 13:15:47.045671 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.045642 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b7cd4689d-xqwkw"] Sep 17 13:15:47.048197 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.048159 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-699c65d854-48j7l" podStartSLOduration=2.566285842 podStartE2EDuration="6.048146559s" totalImagesPullingTime="3.481860717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:47.046655079 +0000 UTC m=+55.951223794" watchObservedRunningTime="2026-09-17 13:15:47.048146559 +0000 UTC m=+55.952715236" Sep 17 13:15:47.049949 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:47.049924 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23b0a37_fe33_443a_93e1_d4a22a561e27.slice/crio-35777cbe2e8a70f7b92a4b454c9fd921585ce88f131c39772f31282e23041766 WatchSource:0}: Error finding container 35777cbe2e8a70f7b92a4b454c9fd921585ce88f131c39772f31282e23041766: Status 404 returned error can't find the container with id 35777cbe2e8a70f7b92a4b454c9fd921585ce88f131c39772f31282e23041766 Sep 17 13:15:47.962665 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.962628 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:15:47.987593 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.987547 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b7cd4689d-xqwkw" event={"ID":"c23b0a37-fe33-443a-93e1-d4a22a561e27","Type":"ContainerStarted","Data":"e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a"} Sep 17 13:15:47.987992 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.987604 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b7cd4689d-xqwkw" event={"ID":"c23b0a37-fe33-443a-93e1-d4a22a561e27","Type":"ContainerStarted","Data":"35777cbe2e8a70f7b92a4b454c9fd921585ce88f131c39772f31282e23041766"} Sep 17 13:15:47.988160 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.988137 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:47.993625 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:47.993606 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d77f96749-ctbf6" Sep 17 13:15:48.005743 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:48.005699 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b7cd4689d-xqwkw" podStartSLOduration=2.005681055 podStartE2EDuration="2.005681055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:48.004594407 +0000 UTC m=+56.909163101" watchObservedRunningTime="2026-09-17 13:15:48.005681055 +0000 UTC m=+56.910249750" Sep 17 13:15:49.773395 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:49.773367 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wh9bs" Sep 17 13:15:51.644102 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.644072 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b7cd4689d-xqwkw"] Sep 17 13:15:51.677769 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.677739 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fb5fb5bcb-2kzrb"] Sep 17 13:15:51.680966 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.680946 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.690452 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.690432 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fb5fb5bcb-2kzrb"] Sep 17 13:15:51.787675 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.787646 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsktx\" (UniqueName: \"kubernetes.io/projected/2a0da9a2-3031-4efd-9b25-205148453681-kube-api-access-nsktx\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.787675 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.787675 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-oauth-serving-cert\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.787841 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.787760 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-console-config\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.787841 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.787802 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-serving-cert\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.787841 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.787821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-service-ca\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.787954 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.787848 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-trusted-ca-bundle\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.787954 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.787886 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-oauth-config\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.888395 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.888366 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsktx\" (UniqueName: \"kubernetes.io/projected/2a0da9a2-3031-4efd-9b25-205148453681-kube-api-access-nsktx\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.888519 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.888396 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-oauth-serving-cert\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.888519 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.888441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-console-config\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.888519 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.888486 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-serving-cert\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.888519 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.888510 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-service-ca\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.888696 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.888539 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-trusted-ca-bundle\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.889009 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.888570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-oauth-config\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.889196 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.889174 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-oauth-serving-cert\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.889465 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.889226 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-console-config\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.889465 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.889317 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-service-ca\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.889801 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.889774 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-trusted-ca-bundle\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.891203 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.891177 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-serving-cert\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.891295 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.891265 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-oauth-config\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.896857 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.896804 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsktx\" (UniqueName: \"kubernetes.io/projected/2a0da9a2-3031-4efd-9b25-205148453681-kube-api-access-nsktx\") pod \"console-5fb5fb5bcb-2kzrb\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:51.989926 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:51.989901 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:15:52.261595 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:52.261507 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fb5fb5bcb-2kzrb"] Sep 17 13:15:52.264049 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:52.264022 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0da9a2_3031_4efd_9b25_205148453681.slice/crio-a58e969ca1c3933e8ad13006282425cf8234366a2e981f5af101082e23f7a53f WatchSource:0}: Error finding container a58e969ca1c3933e8ad13006282425cf8234366a2e981f5af101082e23f7a53f: Status 404 returned error can't find the container with id a58e969ca1c3933e8ad13006282425cf8234366a2e981f5af101082e23f7a53f Sep 17 13:15:53.009936 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:53.009910 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb5fb5bcb-2kzrb" event={"ID":"2a0da9a2-3031-4efd-9b25-205148453681","Type":"ContainerStarted","Data":"69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443"} Sep 17 13:15:53.010321 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:53.009951 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb5fb5bcb-2kzrb" event={"ID":"2a0da9a2-3031-4efd-9b25-205148453681","Type":"ContainerStarted","Data":"a58e969ca1c3933e8ad13006282425cf8234366a2e981f5af101082e23f7a53f"} Sep 17 13:15:53.032337 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:53.032295 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fb5fb5bcb-2kzrb" podStartSLOduration=2.032265439 podStartE2EDuration="2.032265439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:15:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:15:53.029962853 +0000 UTC m=+61.934531544" watchObservedRunningTime="2026-09-17 13:15:53.032265439 +0000 UTC m=+61.936834116" Sep 17 13:15:56.772559 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:56.772524 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:15:57.335744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.335711 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:57.337982 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.337966 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 13:15:57.348216 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.348201 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 13:15:57.359052 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.359024 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kq6t\" (UniqueName: \"kubernetes.io/projected/cca57ec8-2a52-4896-a70d-fdd82708f7f4-kube-api-access-9kq6t\") pod \"network-check-target-2f8lr\" (UID: \"cca57ec8-2a52-4896-a70d-fdd82708f7f4\") " pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:57.425153 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.425130 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kgd8g\"" Sep 17 13:15:57.433616 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.433600 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:15:57.436309 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.436290 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:57.438376 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.438362 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 13:15:57.448970 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.448942 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084-original-pull-secret\") pod \"global-pull-secret-syncer-p4p6z\" (UID: \"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084\") " pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:57.702762 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.702737 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2f8lr"] Sep 17 13:15:57.705109 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:57.705084 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca57ec8_2a52_4896_a70d_fdd82708f7f4.slice/crio-b46169612b35ffd31d31770da9b21a6a90929afaf15cc27c8b2156dff2e50d8e WatchSource:0}: Error finding container b46169612b35ffd31d31770da9b21a6a90929afaf15cc27c8b2156dff2e50d8e: Status 404 returned error can't find the container with id b46169612b35ffd31d31770da9b21a6a90929afaf15cc27c8b2156dff2e50d8e Sep 17 13:15:57.716193 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.716174 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4p6z" Sep 17 13:15:57.983865 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:57.983784 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p4p6z"] Sep 17 13:15:57.986720 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:15:57.986687 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccee0a47_fcb3_4f7c_a9d5_1cb84a6de084.slice/crio-ffba8d79788def6ba44f8305bcad248135c96d714f463c3de21bf6d0bea5c5e7 WatchSource:0}: Error finding container ffba8d79788def6ba44f8305bcad248135c96d714f463c3de21bf6d0bea5c5e7: Status 404 returned error can't find the container with id ffba8d79788def6ba44f8305bcad248135c96d714f463c3de21bf6d0bea5c5e7 Sep 17 13:15:58.030151 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:58.030117 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p4p6z" event={"ID":"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084","Type":"ContainerStarted","Data":"ffba8d79788def6ba44f8305bcad248135c96d714f463c3de21bf6d0bea5c5e7"} Sep 17 13:15:58.031236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:15:58.031214 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2f8lr" event={"ID":"cca57ec8-2a52-4896-a70d-fdd82708f7f4","Type":"ContainerStarted","Data":"b46169612b35ffd31d31770da9b21a6a90929afaf15cc27c8b2156dff2e50d8e"} Sep 17 13:16:00.797998 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:00.797954 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:16:00.798426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:00.798027 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:16:01.990558 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:01.990517 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:16:01.990903 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:01.990596 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:16:01.995221 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:01.995194 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:16:02.054112 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:02.054087 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:16:02.105517 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:02.105491 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68948f6644-rjbjn"] Sep 17 13:16:04.060160 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:04.060125 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p4p6z" event={"ID":"ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084","Type":"ContainerStarted","Data":"35c472e859f52d0e6cef1a92b2843783d298a7f91e3ace1cf0dd6207ecfd7431"} Sep 17 13:16:04.061859 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:04.061835 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2f8lr" event={"ID":"cca57ec8-2a52-4896-a70d-fdd82708f7f4","Type":"ContainerStarted","Data":"e8b30418d77d154b4aae111a561178f0bb5b0c7f0d8022d901056c74c1f5758d"} Sep 17 13:16:04.061982 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:04.061918 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:16:04.076852 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:04.076699 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p4p6z" podStartSLOduration=66.280010122 podStartE2EDuration="1m12.076683316s" totalImagesPullingTime="5.796673194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:16:04.076375214 +0000 UTC m=+72.980943892" watchObservedRunningTime="2026-09-17 13:16:04.076683316 +0000 UTC m=+72.981251994" Sep 17 13:16:04.091762 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:04.091720 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2f8lr" podStartSLOduration=66.024578101 podStartE2EDuration="1m12.091708251s" totalImagesPullingTime="6.06713015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:16:04.090407495 +0000 UTC m=+72.994976171" watchObservedRunningTime="2026-09-17 13:16:04.091708251 +0000 UTC m=+72.996276928" Sep 17 13:16:11.457472 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.457430 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c4dd4b857-98m6b" podUID="467a959a-d853-4d6a-b63c-d0b1b5348848" containerName="console" containerID="cri-o://7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4" gracePeriod=15 Sep 17 13:16:11.697224 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.697201 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:16:11.764661 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.764638 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2khx\" (UniqueName: \"kubernetes.io/projected/467a959a-d853-4d6a-b63c-d0b1b5348848-kube-api-access-k2khx\") pod \"467a959a-d853-4d6a-b63c-d0b1b5348848\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " Sep 17 13:16:11.764798 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.764670 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-oauth-serving-cert\") pod \"467a959a-d853-4d6a-b63c-d0b1b5348848\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " Sep 17 13:16:11.764798 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.764691 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-console-config\") pod \"467a959a-d853-4d6a-b63c-d0b1b5348848\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " Sep 17 13:16:11.764798 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.764719 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-oauth-config\") pod \"467a959a-d853-4d6a-b63c-d0b1b5348848\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " Sep 17 13:16:11.764798 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.764747 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-serving-cert\") pod \"467a959a-d853-4d6a-b63c-d0b1b5348848\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " Sep 17 13:16:11.764798 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.764765 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-service-ca\") pod \"467a959a-d853-4d6a-b63c-d0b1b5348848\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " Sep 17 13:16:11.764798 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.764781 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-trusted-ca-bundle\") pod \"467a959a-d853-4d6a-b63c-d0b1b5348848\" (UID: \"467a959a-d853-4d6a-b63c-d0b1b5348848\") " Sep 17 13:16:11.765142 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.765084 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "467a959a-d853-4d6a-b63c-d0b1b5348848" (UID: "467a959a-d853-4d6a-b63c-d0b1b5348848"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:11.765204 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.765163 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-console-config" (OuterVolumeSpecName: "console-config") pod "467a959a-d853-4d6a-b63c-d0b1b5348848" (UID: "467a959a-d853-4d6a-b63c-d0b1b5348848"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:11.765204 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.765170 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-service-ca" (OuterVolumeSpecName: "service-ca") pod "467a959a-d853-4d6a-b63c-d0b1b5348848" (UID: "467a959a-d853-4d6a-b63c-d0b1b5348848"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:11.765388 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.765359 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "467a959a-d853-4d6a-b63c-d0b1b5348848" (UID: "467a959a-d853-4d6a-b63c-d0b1b5348848"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:11.767022 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.767000 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/467a959a-d853-4d6a-b63c-d0b1b5348848-kube-api-access-k2khx" (OuterVolumeSpecName: "kube-api-access-k2khx") pod "467a959a-d853-4d6a-b63c-d0b1b5348848" (UID: "467a959a-d853-4d6a-b63c-d0b1b5348848"). InnerVolumeSpecName "kube-api-access-k2khx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:16:11.767094 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.767058 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "467a959a-d853-4d6a-b63c-d0b1b5348848" (UID: "467a959a-d853-4d6a-b63c-d0b1b5348848"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:11.767530 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.767513 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "467a959a-d853-4d6a-b63c-d0b1b5348848" (UID: "467a959a-d853-4d6a-b63c-d0b1b5348848"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:11.865494 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.865467 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2khx\" (UniqueName: \"kubernetes.io/projected/467a959a-d853-4d6a-b63c-d0b1b5348848-kube-api-access-k2khx\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:11.865494 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.865490 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-oauth-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:11.865662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.865502 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-console-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:11.865662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.865512 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-oauth-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:11.865662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.865520 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/467a959a-d853-4d6a-b63c-d0b1b5348848-console-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:11.865662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.865528 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-service-ca\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:11.865662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:11.865537 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/467a959a-d853-4d6a-b63c-d0b1b5348848-trusted-ca-bundle\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:12.094073 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.093993 2575 generic.go:356] "Generic (PLEG): container finished" podID="467a959a-d853-4d6a-b63c-d0b1b5348848" containerID="7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4" exitCode=0 Sep 17 13:16:12.094073 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.094040 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4dd4b857-98m6b" Sep 17 13:16:12.094328 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.094087 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4dd4b857-98m6b" event={"ID":"467a959a-d853-4d6a-b63c-d0b1b5348848","Type":"ContainerDied","Data":"7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4"} Sep 17 13:16:12.094328 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.094153 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4dd4b857-98m6b" event={"ID":"467a959a-d853-4d6a-b63c-d0b1b5348848","Type":"ContainerDied","Data":"31c5b8e9fec17e565da5d673ce3d54dd4d1a9f72473d10c87253403fe373a21b"} Sep 17 13:16:12.094328 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.094180 2575 scope.go:117] "RemoveContainer" containerID="7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4" Sep 17 13:16:12.104049 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.104027 2575 scope.go:117] "RemoveContainer" containerID="7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4" Sep 17 13:16:12.104331 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:12.104312 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4\": container with ID starting with 7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4 not found: ID does not exist" containerID="7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4" Sep 17 13:16:12.104395 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.104343 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4"} err="failed to get container status \"7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4\": rpc error: code = NotFound desc = could not find container \"7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4\": container with ID starting with 7204f80d96fd96126bfcdeb98854790735d717a1b2dfdeed757c87e794ba2bb4 not found: ID does not exist" Sep 17 13:16:12.115122 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.115096 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c4dd4b857-98m6b"] Sep 17 13:16:12.119259 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:12.119240 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c4dd4b857-98m6b"] Sep 17 13:16:13.607812 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:13.607704 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467a959a-d853-4d6a-b63c-d0b1b5348848" path="/var/lib/kubelet/pods/467a959a-d853-4d6a-b63c-d0b1b5348848/volumes" Sep 17 13:16:16.663668 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:16.663624 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b7cd4689d-xqwkw" podUID="c23b0a37-fe33-443a-93e1-d4a22a561e27" containerName="console" containerID="cri-o://e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a" gracePeriod=15 Sep 17 13:16:16.928850 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:16.928789 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:16:17.006449 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006420 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-oauth-serving-cert\") pod \"c23b0a37-fe33-443a-93e1-d4a22a561e27\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " Sep 17 13:16:17.006608 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006455 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-config\") pod \"c23b0a37-fe33-443a-93e1-d4a22a561e27\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " Sep 17 13:16:17.006608 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006492 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-serving-cert\") pod \"c23b0a37-fe33-443a-93e1-d4a22a561e27\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " Sep 17 13:16:17.006717 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006617 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-oauth-config\") pod \"c23b0a37-fe33-443a-93e1-d4a22a561e27\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " Sep 17 13:16:17.006717 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006660 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dphtm\" (UniqueName: \"kubernetes.io/projected/c23b0a37-fe33-443a-93e1-d4a22a561e27-kube-api-access-dphtm\") pod \"c23b0a37-fe33-443a-93e1-d4a22a561e27\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " Sep 17 13:16:17.006814 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006725 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-trusted-ca-bundle\") pod \"c23b0a37-fe33-443a-93e1-d4a22a561e27\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " Sep 17 13:16:17.006814 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006797 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-service-ca\") pod \"c23b0a37-fe33-443a-93e1-d4a22a561e27\" (UID: \"c23b0a37-fe33-443a-93e1-d4a22a561e27\") " Sep 17 13:16:17.006905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006827 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c23b0a37-fe33-443a-93e1-d4a22a561e27" (UID: "c23b0a37-fe33-443a-93e1-d4a22a561e27"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:17.006905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.006825 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-config" (OuterVolumeSpecName: "console-config") pod "c23b0a37-fe33-443a-93e1-d4a22a561e27" (UID: "c23b0a37-fe33-443a-93e1-d4a22a561e27"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:17.007062 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.007036 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-oauth-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:17.007218 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.007070 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:17.007218 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.007184 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c23b0a37-fe33-443a-93e1-d4a22a561e27" (UID: "c23b0a37-fe33-443a-93e1-d4a22a561e27"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:17.007485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.007255 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-service-ca" (OuterVolumeSpecName: "service-ca") pod "c23b0a37-fe33-443a-93e1-d4a22a561e27" (UID: "c23b0a37-fe33-443a-93e1-d4a22a561e27"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:17.008862 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.008833 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c23b0a37-fe33-443a-93e1-d4a22a561e27" (UID: "c23b0a37-fe33-443a-93e1-d4a22a561e27"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:17.008950 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.008876 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23b0a37-fe33-443a-93e1-d4a22a561e27-kube-api-access-dphtm" (OuterVolumeSpecName: "kube-api-access-dphtm") pod "c23b0a37-fe33-443a-93e1-d4a22a561e27" (UID: "c23b0a37-fe33-443a-93e1-d4a22a561e27"). InnerVolumeSpecName "kube-api-access-dphtm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:16:17.008950 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.008879 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c23b0a37-fe33-443a-93e1-d4a22a561e27" (UID: "c23b0a37-fe33-443a-93e1-d4a22a561e27"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:17.108385 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.108362 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-trusted-ca-bundle\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:17.108385 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.108383 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b0a37-fe33-443a-93e1-d4a22a561e27-service-ca\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:17.108553 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.108391 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:17.108553 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.108399 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b0a37-fe33-443a-93e1-d4a22a561e27-console-oauth-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:17.108553 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.108407 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dphtm\" (UniqueName: \"kubernetes.io/projected/c23b0a37-fe33-443a-93e1-d4a22a561e27-kube-api-access-dphtm\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:17.117927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.117906 2575 generic.go:356] "Generic (PLEG): container finished" podID="c23b0a37-fe33-443a-93e1-d4a22a561e27" containerID="e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a" exitCode=0 Sep 17 13:16:17.118021 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.117949 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b7cd4689d-xqwkw" Sep 17 13:16:17.118021 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.117963 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b7cd4689d-xqwkw" event={"ID":"c23b0a37-fe33-443a-93e1-d4a22a561e27","Type":"ContainerDied","Data":"e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a"} Sep 17 13:16:17.118098 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.118026 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b7cd4689d-xqwkw" event={"ID":"c23b0a37-fe33-443a-93e1-d4a22a561e27","Type":"ContainerDied","Data":"35777cbe2e8a70f7b92a4b454c9fd921585ce88f131c39772f31282e23041766"} Sep 17 13:16:17.118098 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.118045 2575 scope.go:117] "RemoveContainer" containerID="e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a" Sep 17 13:16:17.126418 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.126402 2575 scope.go:117] "RemoveContainer" containerID="e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a" Sep 17 13:16:17.126675 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:17.126655 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a\": container with ID starting with e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a not found: ID does not exist" containerID="e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a" Sep 17 13:16:17.126732 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.126686 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a"} err="failed to get container status \"e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a\": rpc error: code = NotFound desc = could not find container \"e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a\": container with ID starting with e744319b33fb49f85afa0394cd935f32b0ff37992825baa0712f182cca0e694a not found: ID does not exist" Sep 17 13:16:17.141121 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.141075 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b7cd4689d-xqwkw"] Sep 17 13:16:17.150435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.150411 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b7cd4689d-xqwkw"] Sep 17 13:16:17.607759 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:17.607731 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c23b0a37-fe33-443a-93e1-d4a22a561e27" path="/var/lib/kubelet/pods/c23b0a37-fe33-443a-93e1-d4a22a561e27/volumes" Sep 17 13:16:20.802683 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:20.802652 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:16:20.806408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:20.806386 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6565d9b67d-kcl5n" Sep 17 13:16:27.127675 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.127608 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68948f6644-rjbjn" podUID="6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" containerName="console" containerID="cri-o://b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b" gracePeriod=15 Sep 17 13:16:27.354382 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.354360 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:16:27.384945 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.384891 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2sql\" (UniqueName: \"kubernetes.io/projected/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-kube-api-access-t2sql\") pod \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " Sep 17 13:16:27.384945 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.384920 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-oauth-config\") pod \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " Sep 17 13:16:27.385103 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.384948 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-serving-cert\") pod \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " Sep 17 13:16:27.385103 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.384983 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-service-ca\") pod \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " Sep 17 13:16:27.385103 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.385018 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-config\") pod \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " Sep 17 13:16:27.385103 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.385038 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-oauth-serving-cert\") pod \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\" (UID: \"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17\") " Sep 17 13:16:27.385401 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.385372 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-config" (OuterVolumeSpecName: "console-config") pod "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" (UID: "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:27.385518 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.385400 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" (UID: "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:27.385746 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.385514 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" (UID: "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:27.387149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.387126 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" (UID: "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:27.387149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.387139 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-kube-api-access-t2sql" (OuterVolumeSpecName: "kube-api-access-t2sql") pod "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" (UID: "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17"). InnerVolumeSpecName "kube-api-access-t2sql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:16:27.387267 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.387183 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" (UID: "6a77ec8b-def7-43f9-bc3e-c2fad3c64b17"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:27.486619 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.486590 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:27.486619 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.486615 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-oauth-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:27.486619 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.486624 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2sql\" (UniqueName: \"kubernetes.io/projected/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-kube-api-access-t2sql\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:27.486797 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.486633 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-oauth-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:27.486797 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.486642 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-console-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:27.486797 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:27.486649 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17-service-ca\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:28.166517 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.166480 2575 generic.go:356] "Generic (PLEG): container finished" podID="6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" containerID="b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b" exitCode=0 Sep 17 13:16:28.166930 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.166524 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68948f6644-rjbjn" Sep 17 13:16:28.166930 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.166566 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68948f6644-rjbjn" event={"ID":"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17","Type":"ContainerDied","Data":"b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b"} Sep 17 13:16:28.166930 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.166610 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68948f6644-rjbjn" event={"ID":"6a77ec8b-def7-43f9-bc3e-c2fad3c64b17","Type":"ContainerDied","Data":"d71817b9dbc66bc32b05fcb4e7cdad65498de2ab8ffdbca004981c5473b19d93"} Sep 17 13:16:28.166930 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.166631 2575 scope.go:117] "RemoveContainer" containerID="b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b" Sep 17 13:16:28.174698 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.174683 2575 scope.go:117] "RemoveContainer" containerID="b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b" Sep 17 13:16:28.174975 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:28.174957 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b\": container with ID starting with b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b not found: ID does not exist" containerID="b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b" Sep 17 13:16:28.175018 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.174982 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b"} err="failed to get container status \"b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b\": rpc error: code = NotFound desc = could not find container \"b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b\": container with ID starting with b4d5ce7c3a11b03959a21a9a66c9cc1f4696fbe6d7bb8100c99c76c2bf33005b not found: ID does not exist" Sep 17 13:16:28.187182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.187163 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68948f6644-rjbjn"] Sep 17 13:16:28.194431 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:28.194406 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68948f6644-rjbjn"] Sep 17 13:16:29.607587 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:29.607549 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" path="/var/lib/kubelet/pods/6a77ec8b-def7-43f9-bc3e-c2fad3c64b17/volumes" Sep 17 13:16:35.068614 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:35.068580 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2f8lr" Sep 17 13:16:56.554099 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:56.554066 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:16:56.554562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:56.554520 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="alertmanager" containerID="cri-o://d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3" gracePeriod=120 Sep 17 13:16:56.554875 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:56.554820 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy" containerID="cri-o://d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b" gracePeriod=120 Sep 17 13:16:56.555026 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:56.554986 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy-web" containerID="cri-o://20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9" gracePeriod=120 Sep 17 13:16:56.555094 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:56.555037 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy-metric" containerID="cri-o://99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef" gracePeriod=120 Sep 17 13:16:56.555094 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:56.555011 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="prom-label-proxy" containerID="cri-o://58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2" gracePeriod=120 Sep 17 13:16:56.555822 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:56.554848 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="config-reloader" containerID="cri-o://063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17" gracePeriod=120 Sep 17 13:16:57.280992 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.280959 2575 generic.go:356] "Generic (PLEG): container finished" podID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerID="58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2" exitCode=0 Sep 17 13:16:57.280992 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.280986 2575 generic.go:356] "Generic (PLEG): container finished" podID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerID="d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b" exitCode=0 Sep 17 13:16:57.280992 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.280996 2575 generic.go:356] "Generic (PLEG): container finished" podID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerID="20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9" exitCode=0 Sep 17 13:16:57.280992 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.281003 2575 generic.go:356] "Generic (PLEG): container finished" podID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerID="063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17" exitCode=0 Sep 17 13:16:57.281340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.281009 2575 generic.go:356] "Generic (PLEG): container finished" podID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerID="d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3" exitCode=0 Sep 17 13:16:57.281340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.281025 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerDied","Data":"58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2"} Sep 17 13:16:57.281340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.281086 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerDied","Data":"d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b"} Sep 17 13:16:57.281340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.281111 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerDied","Data":"20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9"} Sep 17 13:16:57.281340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.281135 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerDied","Data":"063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17"} Sep 17 13:16:57.281340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.281156 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerDied","Data":"d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3"} Sep 17 13:16:57.814258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.814237 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:57.937265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937194 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-web\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937222 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-config-volume\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937257 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937301 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937329 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-main-db\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937344 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-main-tls\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937358 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-metrics-client-ca\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937387 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-cluster-tls-config\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937445 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-web-config\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937470 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-tls-assets\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937505 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-config-out\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937527 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.937560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937563 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r52dn\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-kube-api-access-r52dn\") pod \"68420cdc-3117-4eb5-a71e-da01d54244ad\" (UID: \"68420cdc-3117-4eb5-a71e-da01d54244ad\") " Sep 17 13:16:57.938203 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937686 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:16:57.938203 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937706 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:57.938203 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937851 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-main-db\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:57.938203 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.937873 2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-metrics-client-ca\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:57.938954 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.938888 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:16:57.940416 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.940391 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:57.940779 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.940745 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-kube-api-access-r52dn" (OuterVolumeSpecName: "kube-api-access-r52dn") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "kube-api-access-r52dn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:16:57.940886 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.940777 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:57.940956 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.940904 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-config-out" (OuterVolumeSpecName: "config-out") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:16:57.940956 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.940933 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:57.941183 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.941162 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-config-volume" (OuterVolumeSpecName: "config-volume") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:57.941299 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.941246 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:57.941390 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.941375 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:16:57.946060 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.946041 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:57.952455 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:57.952434 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-web-config" (OuterVolumeSpecName: "web-config") pod "68420cdc-3117-4eb5-a71e-da01d54244ad" (UID: "68420cdc-3117-4eb5-a71e-da01d54244ad"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:16:58.039010 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.038986 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039010 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039008 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-main-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039017 2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-cluster-tls-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039043 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-web-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039051 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-tls-assets\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039059 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68420cdc-3117-4eb5-a71e-da01d54244ad-config-out\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039066 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039083 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r52dn\" (UniqueName: \"kubernetes.io/projected/68420cdc-3117-4eb5-a71e-da01d54244ad-kube-api-access-r52dn\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039092 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039099 2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68420cdc-3117-4eb5-a71e-da01d54244ad-config-volume\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.039141 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.039107 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68420cdc-3117-4eb5-a71e-da01d54244ad-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:16:58.287853 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.287823 2575 generic.go:356] "Generic (PLEG): container finished" podID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerID="99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef" exitCode=0 Sep 17 13:16:58.288021 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.287883 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerDied","Data":"99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef"} Sep 17 13:16:58.288021 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.287895 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.288021 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.287954 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68420cdc-3117-4eb5-a71e-da01d54244ad","Type":"ContainerDied","Data":"7535748066d43438eb1e14ef7b24e1286c69cafb88680323825f4b1b0a4fc391"} Sep 17 13:16:58.288021 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.288000 2575 scope.go:117] "RemoveContainer" containerID="58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2" Sep 17 13:16:58.296151 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.296132 2575 scope.go:117] "RemoveContainer" containerID="99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef" Sep 17 13:16:58.303001 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.302987 2575 scope.go:117] "RemoveContainer" containerID="d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b" Sep 17 13:16:58.309752 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.309690 2575 scope.go:117] "RemoveContainer" containerID="20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9" Sep 17 13:16:58.310736 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.310715 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:16:58.314618 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.314598 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:16:58.318669 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.318650 2575 scope.go:117] "RemoveContainer" containerID="063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17" Sep 17 13:16:58.325134 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.325116 2575 scope.go:117] "RemoveContainer" containerID="d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3" Sep 17 13:16:58.331738 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.331722 2575 scope.go:117] "RemoveContainer" containerID="fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f" Sep 17 13:16:58.338436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.338420 2575 scope.go:117] "RemoveContainer" containerID="58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2" Sep 17 13:16:58.338675 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:58.338659 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2\": container with ID starting with 58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2 not found: ID does not exist" containerID="58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2" Sep 17 13:16:58.338724 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.338686 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2"} err="failed to get container status \"58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2\": rpc error: code = NotFound desc = could not find container \"58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2\": container with ID starting with 58291fb963b51ca8eebb97102210784cd17c229c9468b9ba4c16ba1595f5e2f2 not found: ID does not exist" Sep 17 13:16:58.338724 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.338710 2575 scope.go:117] "RemoveContainer" containerID="99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef" Sep 17 13:16:58.338946 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:58.338931 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef\": container with ID starting with 99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef not found: ID does not exist" containerID="99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef" Sep 17 13:16:58.338992 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.338952 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef"} err="failed to get container status \"99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef\": rpc error: code = NotFound desc = could not find container \"99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef\": container with ID starting with 99a5a7bf83e9ef6746b58f43ea65ec0da019ad79b5d44543caba42c012119eef not found: ID does not exist" Sep 17 13:16:58.338992 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.338966 2575 scope.go:117] "RemoveContainer" containerID="d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b" Sep 17 13:16:58.339197 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:58.339181 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b\": container with ID starting with d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b not found: ID does not exist" containerID="d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b" Sep 17 13:16:58.339244 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.339203 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b"} err="failed to get container status \"d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b\": rpc error: code = NotFound desc = could not find container \"d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b\": container with ID starting with d02fdf1b933ac9503770ad3ab05388e04a72096904b739e155f3591845d5868b not found: ID does not exist" Sep 17 13:16:58.339244 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.339219 2575 scope.go:117] "RemoveContainer" containerID="20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9" Sep 17 13:16:58.339486 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:58.339469 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9\": container with ID starting with 20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9 not found: ID does not exist" containerID="20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9" Sep 17 13:16:58.339579 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.339495 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9"} err="failed to get container status \"20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9\": rpc error: code = NotFound desc = could not find container \"20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9\": container with ID starting with 20d57ffa8b0b447857fc1ebc125f881409c085b7735df122f45c04eca05378e9 not found: ID does not exist" Sep 17 13:16:58.339579 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.339556 2575 scope.go:117] "RemoveContainer" containerID="063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17" Sep 17 13:16:58.340042 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:58.340019 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17\": container with ID starting with 063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17 not found: ID does not exist" containerID="063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17" Sep 17 13:16:58.340133 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.340048 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17"} err="failed to get container status \"063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17\": rpc error: code = NotFound desc = could not find container \"063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17\": container with ID starting with 063779d33d6571d926d4f875142e8b8e4338a0fb1b7e9a6f9423364e135f9b17 not found: ID does not exist" Sep 17 13:16:58.340133 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.340079 2575 scope.go:117] "RemoveContainer" containerID="d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3" Sep 17 13:16:58.340432 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:58.340412 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3\": container with ID starting with d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3 not found: ID does not exist" containerID="d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3" Sep 17 13:16:58.340519 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.340436 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3"} err="failed to get container status \"d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3\": rpc error: code = NotFound desc = could not find container \"d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3\": container with ID starting with d60050c57ec251f5ce1a58fbd9adf66f856c44c3057e0a3002c3bf821d1821c3 not found: ID does not exist" Sep 17 13:16:58.340519 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.340453 2575 scope.go:117] "RemoveContainer" containerID="fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f" Sep 17 13:16:58.340762 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:16:58.340746 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f\": container with ID starting with fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f not found: ID does not exist" containerID="fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f" Sep 17 13:16:58.340808 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.340767 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f"} err="failed to get container status \"fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f\": rpc error: code = NotFound desc = could not find container \"fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f\": container with ID starting with fe28d0e93228b3b41bab804619768a1a05249b81d9ec9963c9f06421cf2ad85f not found: ID does not exist" Sep 17 13:16:58.341555 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341540 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:16:58.341807 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341797 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="prom-label-proxy" Sep 17 13:16:58.341844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341809 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="prom-label-proxy" Sep 17 13:16:58.341844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341819 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23b0a37-fe33-443a-93e1-d4a22a561e27" containerName="console" Sep 17 13:16:58.341844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341824 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23b0a37-fe33-443a-93e1-d4a22a561e27" containerName="console" Sep 17 13:16:58.341844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341832 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" containerName="console" Sep 17 13:16:58.341844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341838 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" containerName="console" Sep 17 13:16:58.341844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341843 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467a959a-d853-4d6a-b63c-d0b1b5348848" containerName="console" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341848 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="467a959a-d853-4d6a-b63c-d0b1b5348848" containerName="console" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341859 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="config-reloader" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341864 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="config-reloader" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341869 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy-metric" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341874 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy-metric" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341881 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="alertmanager" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341885 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="alertmanager" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341891 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy-web" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341896 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy-web" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341903 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341908 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341915 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="init-config-reloader" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341920 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="init-config-reloader" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341958 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy-metric" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341968 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="config-reloader" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341975 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c23b0a37-fe33-443a-93e1-d4a22a561e27" containerName="console" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341980 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="467a959a-d853-4d6a-b63c-d0b1b5348848" containerName="console" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341985 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="alertmanager" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341990 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="prom-label-proxy" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.341996 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy-web" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.342001 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a77ec8b-def7-43f9-bc3e-c2fad3c64b17" containerName="console" Sep 17 13:16:58.342019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.342007 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" containerName="kube-rbac-proxy" Sep 17 13:16:58.345036 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.345021 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.347260 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347244 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 17 13:16:58.347260 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347250 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 17 13:16:58.347489 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347471 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 17 13:16:58.347605 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347494 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 17 13:16:58.347605 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347498 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 17 13:16:58.347605 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347479 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 17 13:16:58.347791 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347774 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 17 13:16:58.347854 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347797 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9d445\"" Sep 17 13:16:58.347854 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.347830 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 17 13:16:58.352876 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.352858 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 17 13:16:58.360031 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.360012 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:16:58.442100 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442075 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/256c53e3-30b9-4a95-b585-c7788d5f4fe5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442108 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442131 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/256c53e3-30b9-4a95-b585-c7788d5f4fe5-config-out\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/256c53e3-30b9-4a95-b585-c7788d5f4fe5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442225 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/256c53e3-30b9-4a95-b585-c7788d5f4fe5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442256 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442298 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/256c53e3-30b9-4a95-b585-c7788d5f4fe5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442319 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442339 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-config-volume\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442362 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4m5p\" (UniqueName: \"kubernetes.io/projected/256c53e3-30b9-4a95-b585-c7788d5f4fe5-kube-api-access-w4m5p\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442410 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442589 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442448 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-web-config\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.442589 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.442470 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543527 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543459 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543527 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/256c53e3-30b9-4a95-b585-c7788d5f4fe5-config-out\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543527 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543521 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/256c53e3-30b9-4a95-b585-c7788d5f4fe5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/256c53e3-30b9-4a95-b585-c7788d5f4fe5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543565 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543589 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/256c53e3-30b9-4a95-b585-c7788d5f4fe5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543614 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543638 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-config-volume\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543928 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4m5p\" (UniqueName: \"kubernetes.io/projected/256c53e3-30b9-4a95-b585-c7788d5f4fe5-kube-api-access-w4m5p\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543928 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543815 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543928 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543851 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-web-config\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.543928 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543891 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.544124 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.543937 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/256c53e3-30b9-4a95-b585-c7788d5f4fe5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.544405 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.544378 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/256c53e3-30b9-4a95-b585-c7788d5f4fe5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.544503 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.544463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/256c53e3-30b9-4a95-b585-c7788d5f4fe5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.545152 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.545131 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/256c53e3-30b9-4a95-b585-c7788d5f4fe5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.546808 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.546684 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/256c53e3-30b9-4a95-b585-c7788d5f4fe5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.546808 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.546703 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/256c53e3-30b9-4a95-b585-c7788d5f4fe5-config-out\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.546808 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.546754 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.546997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.546812 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.546997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.546832 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.547110 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.547012 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-web-config\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.547350 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.547330 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.547576 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.547559 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-config-volume\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.548641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.548626 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/256c53e3-30b9-4a95-b585-c7788d5f4fe5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.555606 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.555590 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4m5p\" (UniqueName: \"kubernetes.io/projected/256c53e3-30b9-4a95-b585-c7788d5f4fe5-kube-api-access-w4m5p\") pod \"alertmanager-main-0\" (UID: \"256c53e3-30b9-4a95-b585-c7788d5f4fe5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.655327 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.655306 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 13:16:58.939323 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:58.939247 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 13:16:58.941498 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:16:58.941470 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod256c53e3_30b9_4a95_b585_c7788d5f4fe5.slice/crio-f0624ea39b165db8bf172478748ce7bb0f35bc2e6381d77ab8c48d1db15c360e WatchSource:0}: Error finding container f0624ea39b165db8bf172478748ce7bb0f35bc2e6381d77ab8c48d1db15c360e: Status 404 returned error can't find the container with id f0624ea39b165db8bf172478748ce7bb0f35bc2e6381d77ab8c48d1db15c360e Sep 17 13:16:59.293897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:59.293865 2575 generic.go:356] "Generic (PLEG): container finished" podID="256c53e3-30b9-4a95-b585-c7788d5f4fe5" containerID="f32f0d1f0b8709aa86f1b1f01697d18d772de5950f5ad600937403fe1887c8f7" exitCode=0 Sep 17 13:16:59.294050 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:59.293963 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"256c53e3-30b9-4a95-b585-c7788d5f4fe5","Type":"ContainerDied","Data":"f32f0d1f0b8709aa86f1b1f01697d18d772de5950f5ad600937403fe1887c8f7"} Sep 17 13:16:59.294090 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:59.294072 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"256c53e3-30b9-4a95-b585-c7788d5f4fe5","Type":"ContainerStarted","Data":"f0624ea39b165db8bf172478748ce7bb0f35bc2e6381d77ab8c48d1db15c360e"} Sep 17 13:16:59.609435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:16:59.609409 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68420cdc-3117-4eb5-a71e-da01d54244ad" path="/var/lib/kubelet/pods/68420cdc-3117-4eb5-a71e-da01d54244ad/volumes" Sep 17 13:17:00.301856 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:00.301819 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"256c53e3-30b9-4a95-b585-c7788d5f4fe5","Type":"ContainerStarted","Data":"e4ffb304e57ad94b78b665784d339cba563a180d0d2bb345c0a677bc3c8ee7d2"} Sep 17 13:17:00.302211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:00.301881 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"256c53e3-30b9-4a95-b585-c7788d5f4fe5","Type":"ContainerStarted","Data":"3ca774d28dafaa24334384d06c0c34a59b7b3ac8d510b99bdb379a0a69312e87"} Sep 17 13:17:00.302211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:00.301906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"256c53e3-30b9-4a95-b585-c7788d5f4fe5","Type":"ContainerStarted","Data":"fc4db9135b113bb1c6dd55d9dcf36705175e295dfe9a85785cbbce35fdeae9b0"} Sep 17 13:17:00.302211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:00.301930 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"256c53e3-30b9-4a95-b585-c7788d5f4fe5","Type":"ContainerStarted","Data":"f85bf2290419b0ed06f7d69b0d0adcfd8f859153384da74f7226bd397232a044"} Sep 17 13:17:00.302211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:00.301960 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"256c53e3-30b9-4a95-b585-c7788d5f4fe5","Type":"ContainerStarted","Data":"8a783f9107e23eaf5f9932eb92bda4ec27c56100b83b285ba72b26b794b66ed7"} Sep 17 13:17:00.302211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:00.301983 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"256c53e3-30b9-4a95-b585-c7788d5f4fe5","Type":"ContainerStarted","Data":"03d2bcad7e2ae3031a5f98a8661f3f2a3fba1b9b3290153052d599f95823c72f"} Sep 17 13:17:00.334142 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:00.334100 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.320008421 podStartE2EDuration="2.334081033s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.072612ms" isStatefulPod=false podCreationTimestamp="2026-09-17 13:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:17:00.333739427 +0000 UTC m=+129.238308158" watchObservedRunningTime="2026-09-17 13:17:00.334081033 +0000 UTC m=+129.238649712" Sep 17 13:17:01.618666 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.618621 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85f9cd65b6-sm4c7"] Sep 17 13:17:01.621607 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.621587 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.635387 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.635363 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85f9cd65b6-sm4c7"] Sep 17 13:17:01.670735 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.670712 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-config\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.670834 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.670743 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-oauth-config\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.670834 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.670767 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-oauth-serving-cert\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.670905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.670826 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-service-ca\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.670905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.670884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-serving-cert\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.670962 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.670928 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7msk7\" (UniqueName: \"kubernetes.io/projected/fc1c8dda-aed1-48c8-9557-08248c0a2727-kube-api-access-7msk7\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.670994 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.670963 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-trusted-ca-bundle\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.771662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.771614 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7msk7\" (UniqueName: \"kubernetes.io/projected/fc1c8dda-aed1-48c8-9557-08248c0a2727-kube-api-access-7msk7\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.771662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.771675 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-trusted-ca-bundle\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.771887 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.771709 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-config\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.771887 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.771730 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-oauth-config\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.771887 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.771754 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-oauth-serving-cert\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.771887 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.771775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-service-ca\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.772072 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.771928 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-serving-cert\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.772460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.772431 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-config\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.772729 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.772541 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-service-ca\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.772729 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.772541 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-oauth-serving-cert\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.772729 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.772673 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-trusted-ca-bundle\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.774259 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.774238 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-oauth-config\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.774427 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.774412 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-serving-cert\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.780467 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.780447 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7msk7\" (UniqueName: \"kubernetes.io/projected/fc1c8dda-aed1-48c8-9557-08248c0a2727-kube-api-access-7msk7\") pod \"console-85f9cd65b6-sm4c7\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:01.931001 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:01.930921 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:02.203219 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:02.203027 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85f9cd65b6-sm4c7"] Sep 17 13:17:02.205569 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:17:02.205544 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1c8dda_aed1_48c8_9557_08248c0a2727.slice/crio-72fbdc1053870bb0dcb337b4f3441b29a1bb66e4a64061daa456266a4302d61e WatchSource:0}: Error finding container 72fbdc1053870bb0dcb337b4f3441b29a1bb66e4a64061daa456266a4302d61e: Status 404 returned error can't find the container with id 72fbdc1053870bb0dcb337b4f3441b29a1bb66e4a64061daa456266a4302d61e Sep 17 13:17:02.313607 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:02.313572 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f9cd65b6-sm4c7" event={"ID":"fc1c8dda-aed1-48c8-9557-08248c0a2727","Type":"ContainerStarted","Data":"d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb"} Sep 17 13:17:02.313760 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:02.313628 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f9cd65b6-sm4c7" event={"ID":"fc1c8dda-aed1-48c8-9557-08248c0a2727","Type":"ContainerStarted","Data":"72fbdc1053870bb0dcb337b4f3441b29a1bb66e4a64061daa456266a4302d61e"} Sep 17 13:17:02.334298 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:02.334235 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85f9cd65b6-sm4c7" podStartSLOduration=1.33422085 podStartE2EDuration="1.33422085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:17:02.332740403 +0000 UTC m=+131.237309098" watchObservedRunningTime="2026-09-17 13:17:02.33422085 +0000 UTC m=+131.238789527" Sep 17 13:17:11.931387 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:11.931346 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:11.931387 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:11.931402 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:11.935825 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:11.935806 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:12.356759 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:12.356731 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:17:12.406848 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:12.406820 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fb5fb5bcb-2kzrb"] Sep 17 13:17:37.425962 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.425902 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fb5fb5bcb-2kzrb" podUID="2a0da9a2-3031-4efd-9b25-205148453681" containerName="console" containerID="cri-o://69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443" gracePeriod=15 Sep 17 13:17:37.659548 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.659525 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:17:37.766348 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766317 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-console-config\") pod \"2a0da9a2-3031-4efd-9b25-205148453681\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " Sep 17 13:17:37.766348 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766358 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-service-ca\") pod \"2a0da9a2-3031-4efd-9b25-205148453681\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " Sep 17 13:17:37.766580 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766415 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-oauth-serving-cert\") pod \"2a0da9a2-3031-4efd-9b25-205148453681\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " Sep 17 13:17:37.766580 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766472 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsktx\" (UniqueName: \"kubernetes.io/projected/2a0da9a2-3031-4efd-9b25-205148453681-kube-api-access-nsktx\") pod \"2a0da9a2-3031-4efd-9b25-205148453681\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " Sep 17 13:17:37.766580 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766542 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-trusted-ca-bundle\") pod \"2a0da9a2-3031-4efd-9b25-205148453681\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " Sep 17 13:17:37.766580 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766568 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-oauth-config\") pod \"2a0da9a2-3031-4efd-9b25-205148453681\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " Sep 17 13:17:37.766777 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766610 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-serving-cert\") pod \"2a0da9a2-3031-4efd-9b25-205148453681\" (UID: \"2a0da9a2-3031-4efd-9b25-205148453681\") " Sep 17 13:17:37.766828 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766731 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-console-config" (OuterVolumeSpecName: "console-config") pod "2a0da9a2-3031-4efd-9b25-205148453681" (UID: "2a0da9a2-3031-4efd-9b25-205148453681"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:17:37.766881 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766827 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-service-ca" (OuterVolumeSpecName: "service-ca") pod "2a0da9a2-3031-4efd-9b25-205148453681" (UID: "2a0da9a2-3031-4efd-9b25-205148453681"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:17:37.766960 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766899 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-console-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:17:37.766960 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766917 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-service-ca\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:17:37.767061 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.766905 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2a0da9a2-3031-4efd-9b25-205148453681" (UID: "2a0da9a2-3031-4efd-9b25-205148453681"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:17:37.767139 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.767115 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2a0da9a2-3031-4efd-9b25-205148453681" (UID: "2a0da9a2-3031-4efd-9b25-205148453681"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:17:37.769228 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.769197 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2a0da9a2-3031-4efd-9b25-205148453681" (UID: "2a0da9a2-3031-4efd-9b25-205148453681"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:17:37.769377 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.769261 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a0da9a2-3031-4efd-9b25-205148453681-kube-api-access-nsktx" (OuterVolumeSpecName: "kube-api-access-nsktx") pod "2a0da9a2-3031-4efd-9b25-205148453681" (UID: "2a0da9a2-3031-4efd-9b25-205148453681"). InnerVolumeSpecName "kube-api-access-nsktx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:17:37.769377 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.769345 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2a0da9a2-3031-4efd-9b25-205148453681" (UID: "2a0da9a2-3031-4efd-9b25-205148453681"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:17:37.867647 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.867608 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-trusted-ca-bundle\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:17:37.867647 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.867642 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-oauth-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:17:37.867647 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.867650 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2a0da9a2-3031-4efd-9b25-205148453681-console-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:17:37.867870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.867660 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2a0da9a2-3031-4efd-9b25-205148453681-oauth-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:17:37.867870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:37.867668 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsktx\" (UniqueName: \"kubernetes.io/projected/2a0da9a2-3031-4efd-9b25-205148453681-kube-api-access-nsktx\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:17:38.452411 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.452376 2575 generic.go:356] "Generic (PLEG): container finished" podID="2a0da9a2-3031-4efd-9b25-205148453681" containerID="69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443" exitCode=0 Sep 17 13:17:38.452829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.452442 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fb5fb5bcb-2kzrb" Sep 17 13:17:38.452829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.452496 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb5fb5bcb-2kzrb" event={"ID":"2a0da9a2-3031-4efd-9b25-205148453681","Type":"ContainerDied","Data":"69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443"} Sep 17 13:17:38.452829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.452571 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fb5fb5bcb-2kzrb" event={"ID":"2a0da9a2-3031-4efd-9b25-205148453681","Type":"ContainerDied","Data":"a58e969ca1c3933e8ad13006282425cf8234366a2e981f5af101082e23f7a53f"} Sep 17 13:17:38.452829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.452601 2575 scope.go:117] "RemoveContainer" containerID="69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443" Sep 17 13:17:38.462947 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.462925 2575 scope.go:117] "RemoveContainer" containerID="69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443" Sep 17 13:17:38.463210 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:17:38.463191 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443\": container with ID starting with 69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443 not found: ID does not exist" containerID="69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443" Sep 17 13:17:38.463308 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.463218 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443"} err="failed to get container status \"69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443\": rpc error: code = NotFound desc = could not find container \"69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443\": container with ID starting with 69e2405df4fce2172be88e7851fe46430c78584e2603e3fa00ba434447740443 not found: ID does not exist" Sep 17 13:17:38.475463 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.475444 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fb5fb5bcb-2kzrb"] Sep 17 13:17:38.479773 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:38.479752 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fb5fb5bcb-2kzrb"] Sep 17 13:17:39.607703 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:17:39.607671 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a0da9a2-3031-4efd-9b25-205148453681" path="/var/lib/kubelet/pods/2a0da9a2-3031-4efd-9b25-205148453681/volumes" Sep 17 13:19:45.729575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.729542 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx"] Sep 17 13:19:45.729950 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.729832 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a0da9a2-3031-4efd-9b25-205148453681" containerName="console" Sep 17 13:19:45.729950 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.729843 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a0da9a2-3031-4efd-9b25-205148453681" containerName="console" Sep 17 13:19:45.729950 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.729895 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a0da9a2-3031-4efd-9b25-205148453681" containerName="console" Sep 17 13:19:45.732550 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.732533 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:19:45.734418 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.734399 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 13:19:45.734922 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.734901 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 13:19:45.735035 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.734905 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-xnjlv\"" Sep 17 13:19:45.744026 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.744005 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx"] Sep 17 13:19:45.828156 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.828135 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-599r7\" (UniqueName: \"kubernetes.io/projected/59d197d5-ca5e-4828-b158-cb660adc6b33-kube-api-access-599r7\") pod \"feast-operator-controller-manager-68d9d5bf96-t5jqx\" (UID: \"59d197d5-ca5e-4828-b158-cb660adc6b33\") " pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:19:45.929182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.929157 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-599r7\" (UniqueName: \"kubernetes.io/projected/59d197d5-ca5e-4828-b158-cb660adc6b33-kube-api-access-599r7\") pod \"feast-operator-controller-manager-68d9d5bf96-t5jqx\" (UID: \"59d197d5-ca5e-4828-b158-cb660adc6b33\") " pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:19:45.937550 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:45.937528 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-599r7\" (UniqueName: \"kubernetes.io/projected/59d197d5-ca5e-4828-b158-cb660adc6b33-kube-api-access-599r7\") pod \"feast-operator-controller-manager-68d9d5bf96-t5jqx\" (UID: \"59d197d5-ca5e-4828-b158-cb660adc6b33\") " pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:19:46.042741 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:46.042719 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:19:46.308568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:46.308542 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx"] Sep 17 13:19:46.311040 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:19:46.311012 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d197d5_ca5e_4828_b158_cb660adc6b33.slice/crio-a5f87ac099e2f076783ab5270b918f8bf3b7ba06658b1390f381d61683262d21 WatchSource:0}: Error finding container a5f87ac099e2f076783ab5270b918f8bf3b7ba06658b1390f381d61683262d21: Status 404 returned error can't find the container with id a5f87ac099e2f076783ab5270b918f8bf3b7ba06658b1390f381d61683262d21 Sep 17 13:19:46.934706 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:46.934677 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" event={"ID":"59d197d5-ca5e-4828-b158-cb660adc6b33","Type":"ContainerStarted","Data":"a5f87ac099e2f076783ab5270b918f8bf3b7ba06658b1390f381d61683262d21"} Sep 17 13:19:48.944326 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:48.944215 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" event={"ID":"59d197d5-ca5e-4828-b158-cb660adc6b33","Type":"ContainerStarted","Data":"dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953"} Sep 17 13:19:48.944326 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:48.944287 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:19:48.960773 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:48.960725 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" podStartSLOduration=1.6000876979999998 podStartE2EDuration="3.960711673s" totalImagesPullingTime="2.360623975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:19:48.959673802 +0000 UTC m=+297.864242489" watchObservedRunningTime="2026-09-17 13:19:48.960711673 +0000 UTC m=+297.865280350" Sep 17 13:19:51.475903 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:51.475879 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:19:51.480165 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:51.480140 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:19:51.482867 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:51.482849 2575 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 13:19:59.951119 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:19:59.951089 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:20:26.884886 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.884851 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk"] Sep 17 13:20:26.891566 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.891543 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:26.893214 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.893192 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 13:20:26.893333 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.893318 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 13:20:26.893397 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.893347 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 13:20:26.893397 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.893347 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 13:20:26.893397 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.893393 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 13:20:26.893538 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.893348 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 17 13:20:26.893716 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.893699 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 13:20:26.893775 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.893733 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-nkncx\"" Sep 17 13:20:26.899074 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:26.899054 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk"] Sep 17 13:20:27.045895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.045867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-registry-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.046023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.045905 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-offline-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.046023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.045953 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-online-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.046023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.045983 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b9192979-77cb-4a67-9d92-fc9321a549a0-feast-data\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.046023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.046009 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8cwq\" (UniqueName: \"kubernetes.io/projected/b9192979-77cb-4a67-9d92-fc9321a549a0-kube-api-access-w8cwq\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.046023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.046024 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.147302 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.147218 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-registry-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.147302 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.147250 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-offline-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.147473 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.147305 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-online-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.147473 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.147325 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b9192979-77cb-4a67-9d92-fc9321a549a0-feast-data\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.147473 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.147350 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8cwq\" (UniqueName: \"kubernetes.io/projected/b9192979-77cb-4a67-9d92-fc9321a549a0-kube-api-access-w8cwq\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.147473 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.147366 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.147661 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:20:27.147563 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 17 13:20:27.147734 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:20:27.147678 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls podName:b9192979-77cb-4a67-9d92-fc9321a549a0 nodeName:}" failed. No retries permitted until 2026-09-17 13:20:27.647653673 +0000 UTC m=+336.552222345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls") pod "feast-simple-feast-setup-69cd4fb9c9-6s5fk" (UID: "b9192979-77cb-4a67-9d92-fc9321a549a0") : secret "feast-simple-feast-setup-ui-tls" not found Sep 17 13:20:27.147805 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.147765 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b9192979-77cb-4a67-9d92-fc9321a549a0-feast-data\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.149863 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.149840 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-offline-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.149953 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.149868 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-registry-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.149953 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.149892 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-online-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.155150 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.155129 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8cwq\" (UniqueName: \"kubernetes.io/projected/b9192979-77cb-4a67-9d92-fc9321a549a0-kube-api-access-w8cwq\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.650743 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.650715 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.653080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.653053 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-6s5fk\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:27.813206 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:27.813174 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:20:28.094885 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:28.094849 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk"] Sep 17 13:20:28.097898 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:20:28.097868 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9192979_77cb_4a67_9d92_fc9321a549a0.slice/crio-ea8292c2a73502506907e5f0f4c6b01931a5ea0cbddaf01b5c1b185f9e40986b WatchSource:0}: Error finding container ea8292c2a73502506907e5f0f4c6b01931a5ea0cbddaf01b5c1b185f9e40986b: Status 404 returned error can't find the container with id ea8292c2a73502506907e5f0f4c6b01931a5ea0cbddaf01b5c1b185f9e40986b Sep 17 13:20:28.099652 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:28.099639 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 13:20:29.102749 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:29.102708 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerStarted","Data":"ea8292c2a73502506907e5f0f4c6b01931a5ea0cbddaf01b5c1b185f9e40986b"} Sep 17 13:20:30.995071 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:30.995042 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7677987654-qsrns"] Sep 17 13:20:30.998204 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:30.998188 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.007573 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.007544 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7677987654-qsrns"] Sep 17 13:20:31.179614 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.179573 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxp9q\" (UniqueName: \"kubernetes.io/projected/2f9d1295-a1cc-4189-9758-25963bcc9c82-kube-api-access-lxp9q\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.179850 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.179619 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-service-ca\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.179850 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.179650 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-trusted-ca-bundle\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.179850 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.179745 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-serving-cert\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.179850 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.179783 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-config\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.179850 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.179849 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-oauth-config\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.180092 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.179873 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-oauth-serving-cert\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.280740 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.280703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxp9q\" (UniqueName: \"kubernetes.io/projected/2f9d1295-a1cc-4189-9758-25963bcc9c82-kube-api-access-lxp9q\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.280927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.280741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-service-ca\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.280927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.280771 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-trusted-ca-bundle\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.280927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.280804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-serving-cert\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.280927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.280820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-config\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.280927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.280844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-oauth-config\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.280927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.280857 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-oauth-serving-cert\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.281617 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.281593 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-service-ca\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.281721 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.281661 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-oauth-serving-cert\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.281780 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.281761 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-trusted-ca-bundle\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.281814 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.281769 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-config\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.283297 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.283251 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-serving-cert\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.283408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.283386 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f9d1295-a1cc-4189-9758-25963bcc9c82-console-oauth-config\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.289727 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.289699 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxp9q\" (UniqueName: \"kubernetes.io/projected/2f9d1295-a1cc-4189-9758-25963bcc9c82-kube-api-access-lxp9q\") pod \"console-7677987654-qsrns\" (UID: \"2f9d1295-a1cc-4189-9758-25963bcc9c82\") " pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.307242 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.307220 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:31.593979 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:31.593908 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7677987654-qsrns"] Sep 17 13:20:31.597245 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:20:31.597221 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9d1295_a1cc_4189_9758_25963bcc9c82.slice/crio-36a73e2865f5c79a12642f91c7b7e312560812c1ef01476f0d546eafd0e4df62 WatchSource:0}: Error finding container 36a73e2865f5c79a12642f91c7b7e312560812c1ef01476f0d546eafd0e4df62: Status 404 returned error can't find the container with id 36a73e2865f5c79a12642f91c7b7e312560812c1ef01476f0d546eafd0e4df62 Sep 17 13:20:32.115664 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:32.115631 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677987654-qsrns" event={"ID":"2f9d1295-a1cc-4189-9758-25963bcc9c82","Type":"ContainerStarted","Data":"c9e155b81a89a8b244a1909a002e5f1d35d3de9aff8adb2ab4819b6e9d3dc7bf"} Sep 17 13:20:32.116082 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:32.115679 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677987654-qsrns" event={"ID":"2f9d1295-a1cc-4189-9758-25963bcc9c82","Type":"ContainerStarted","Data":"36a73e2865f5c79a12642f91c7b7e312560812c1ef01476f0d546eafd0e4df62"} Sep 17 13:20:32.134018 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:32.133968 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7677987654-qsrns" podStartSLOduration=2.133948448 podStartE2EDuration="2.133948448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:20:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:20:32.131887863 +0000 UTC m=+341.036456550" watchObservedRunningTime="2026-09-17 13:20:32.133948448 +0000 UTC m=+341.038517126" Sep 17 13:20:41.308002 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:41.307966 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:41.308425 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:41.308053 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:41.313855 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:41.313828 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:42.159769 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:42.159745 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7677987654-qsrns" Sep 17 13:20:42.238368 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:42.238332 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85f9cd65b6-sm4c7"] Sep 17 13:20:56.223761 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:20:56.223720 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerStarted","Data":"2ce8f43b89e5ba5c3bb20647692bc349258a9a03b5254f6ef9962506b215f365"} Sep 17 13:21:01.244822 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:01.244792 2575 generic.go:356] "Generic (PLEG): container finished" podID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerID="2ce8f43b89e5ba5c3bb20647692bc349258a9a03b5254f6ef9962506b215f365" exitCode=0 Sep 17 13:21:01.245238 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:01.244873 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerDied","Data":"2ce8f43b89e5ba5c3bb20647692bc349258a9a03b5254f6ef9962506b215f365"} Sep 17 13:21:02.252084 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:02.252051 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerStarted","Data":"f89ceee1d9eb49cc980c6f28a8fe13e81de512c84980f2fa95cb2edfff405d4f"} Sep 17 13:21:07.263722 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.263665 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85f9cd65b6-sm4c7" podUID="fc1c8dda-aed1-48c8-9557-08248c0a2727" containerName="console" containerID="cri-o://d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb" gracePeriod=15 Sep 17 13:21:07.550072 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.550041 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:21:07.697708 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.697678 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-trusted-ca-bundle\") pod \"fc1c8dda-aed1-48c8-9557-08248c0a2727\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " Sep 17 13:21:07.697914 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.697727 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7msk7\" (UniqueName: \"kubernetes.io/projected/fc1c8dda-aed1-48c8-9557-08248c0a2727-kube-api-access-7msk7\") pod \"fc1c8dda-aed1-48c8-9557-08248c0a2727\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " Sep 17 13:21:07.697914 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.697748 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-oauth-serving-cert\") pod \"fc1c8dda-aed1-48c8-9557-08248c0a2727\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " Sep 17 13:21:07.697914 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.697765 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-oauth-config\") pod \"fc1c8dda-aed1-48c8-9557-08248c0a2727\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " Sep 17 13:21:07.697914 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.697883 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-serving-cert\") pod \"fc1c8dda-aed1-48c8-9557-08248c0a2727\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " Sep 17 13:21:07.697914 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.697914 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-service-ca\") pod \"fc1c8dda-aed1-48c8-9557-08248c0a2727\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " Sep 17 13:21:07.698182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.698011 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-config\") pod \"fc1c8dda-aed1-48c8-9557-08248c0a2727\" (UID: \"fc1c8dda-aed1-48c8-9557-08248c0a2727\") " Sep 17 13:21:07.698232 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.698209 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fc1c8dda-aed1-48c8-9557-08248c0a2727" (UID: "fc1c8dda-aed1-48c8-9557-08248c0a2727"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:21:07.698444 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.698425 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-trusted-ca-bundle\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:21:07.698542 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.698442 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-service-ca" (OuterVolumeSpecName: "service-ca") pod "fc1c8dda-aed1-48c8-9557-08248c0a2727" (UID: "fc1c8dda-aed1-48c8-9557-08248c0a2727"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:21:07.698606 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.698539 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fc1c8dda-aed1-48c8-9557-08248c0a2727" (UID: "fc1c8dda-aed1-48c8-9557-08248c0a2727"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:21:07.698606 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.698539 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-config" (OuterVolumeSpecName: "console-config") pod "fc1c8dda-aed1-48c8-9557-08248c0a2727" (UID: "fc1c8dda-aed1-48c8-9557-08248c0a2727"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:21:07.699984 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.699953 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fc1c8dda-aed1-48c8-9557-08248c0a2727" (UID: "fc1c8dda-aed1-48c8-9557-08248c0a2727"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:21:07.700428 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.700404 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fc1c8dda-aed1-48c8-9557-08248c0a2727" (UID: "fc1c8dda-aed1-48c8-9557-08248c0a2727"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:21:07.700535 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.700464 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc1c8dda-aed1-48c8-9557-08248c0a2727-kube-api-access-7msk7" (OuterVolumeSpecName: "kube-api-access-7msk7") pod "fc1c8dda-aed1-48c8-9557-08248c0a2727" (UID: "fc1c8dda-aed1-48c8-9557-08248c0a2727"). InnerVolumeSpecName "kube-api-access-7msk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:21:07.799617 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.799586 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7msk7\" (UniqueName: \"kubernetes.io/projected/fc1c8dda-aed1-48c8-9557-08248c0a2727-kube-api-access-7msk7\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:21:07.799617 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.799618 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-oauth-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:21:07.799812 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.799629 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-oauth-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:21:07.799812 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.799637 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-serving-cert\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:21:07.799812 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.799646 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-service-ca\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:21:07.799812 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:07.799654 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc1c8dda-aed1-48c8-9557-08248c0a2727-console-config\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:21:08.277722 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.277692 2575 generic.go:356] "Generic (PLEG): container finished" podID="fc1c8dda-aed1-48c8-9557-08248c0a2727" containerID="d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb" exitCode=0 Sep 17 13:21:08.278143 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.277742 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f9cd65b6-sm4c7" Sep 17 13:21:08.278143 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.277777 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f9cd65b6-sm4c7" event={"ID":"fc1c8dda-aed1-48c8-9557-08248c0a2727","Type":"ContainerDied","Data":"d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb"} Sep 17 13:21:08.278143 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.277824 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f9cd65b6-sm4c7" event={"ID":"fc1c8dda-aed1-48c8-9557-08248c0a2727","Type":"ContainerDied","Data":"72fbdc1053870bb0dcb337b4f3441b29a1bb66e4a64061daa456266a4302d61e"} Sep 17 13:21:08.278143 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.277848 2575 scope.go:117] "RemoveContainer" containerID="d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb" Sep 17 13:21:08.279996 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.279973 2575 generic.go:356] "Generic (PLEG): container finished" podID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerID="f89ceee1d9eb49cc980c6f28a8fe13e81de512c84980f2fa95cb2edfff405d4f" exitCode=0 Sep 17 13:21:08.280083 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.280010 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerDied","Data":"f89ceee1d9eb49cc980c6f28a8fe13e81de512c84980f2fa95cb2edfff405d4f"} Sep 17 13:21:08.287726 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.287714 2575 scope.go:117] "RemoveContainer" containerID="d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb" Sep 17 13:21:08.288020 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:21:08.287993 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb\": container with ID starting with d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb not found: ID does not exist" containerID="d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb" Sep 17 13:21:08.288122 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.288029 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb"} err="failed to get container status \"d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb\": rpc error: code = NotFound desc = could not find container \"d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb\": container with ID starting with d3d79a148723fe3d55c459fb84d4ef54845a3e2ad5144b68b901d1475d2e31bb not found: ID does not exist" Sep 17 13:21:08.293314 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.292871 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 17 13:21:08.315513 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.315487 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85f9cd65b6-sm4c7"] Sep 17 13:21:08.319821 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:08.319801 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85f9cd65b6-sm4c7"] Sep 17 13:21:09.290245 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.289978 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerStarted","Data":"1538c4287c6380de159e3f885997c9c69b0bb6f06329ced48b008e731624ebbb"} Sep 17 13:21:09.290245 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.290042 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerStarted","Data":"a65231a6fc056b3abd7485779d9fd2a4de6c3f8e5a317e8553790c5f017d4478"} Sep 17 13:21:09.290245 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.290065 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerStarted","Data":"613581b8ff19d607f88d8b5acb7c09cc5d1f20f79db4980d768cdf58a4728480"} Sep 17 13:21:09.290245 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.290082 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerStarted","Data":"f86f3dc9c21dcee201db00711921de6139ae870aeff9fa7acec02190231ec41a"} Sep 17 13:21:09.313216 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.313151 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podStartSLOduration=15.716743968 podStartE2EDuration="43.31312919s" totalImagesPullingTime="27.596385222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:21:09.311593496 +0000 UTC m=+378.216162214" watchObservedRunningTime="2026-09-17 13:21:09.31312919 +0000 UTC m=+378.217697868" Sep 17 13:21:09.611457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.611370 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc1c8dda-aed1-48c8-9557-08248c0a2727" path="/var/lib/kubelet/pods/fc1c8dda-aed1-48c8-9557-08248c0a2727/volumes" Sep 17 13:21:09.814214 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.814095 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:09.814214 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.814160 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:09.814214 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.814179 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:09.814523 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.814297 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:09.816185 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.816039 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 17 13:21:09.816185 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.816095 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 17 13:21:09.816185 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.816104 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 17 13:21:09.816185 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:09.816054 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 17 13:21:12.814523 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:12.814475 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 17 13:21:12.815030 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:12.814475 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 17 13:21:12.815030 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:12.814561 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 17 13:21:12.815030 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:12.814778 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 17 13:21:15.814469 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:15.814422 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 17 13:21:15.815041 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:15.814435 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Sep 17 13:21:15.815160 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:15.814422 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Sep 17 13:21:15.815160 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:15.814553 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 17 13:21:17.814122 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:17.814088 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:17.814498 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:17.814138 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:17.814498 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:17.814150 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:17.814498 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:17.814169 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:18.814393 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:18.814354 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:18.814857 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:18.814658 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:18.814857 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:18.814754 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:18.815526 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:18.815504 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:18.815612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:18.815563 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:18.815767 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:18.815751 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:18.819822 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:18.819806 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:21:19.347861 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:21:19.347827 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:22:21.157251 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.157167 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk"] Sep 17 13:22:21.157729 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.157456 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="registry" containerID="cri-o://f86f3dc9c21dcee201db00711921de6139ae870aeff9fa7acec02190231ec41a" gracePeriod=30 Sep 17 13:22:21.157729 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.157514 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" containerID="cri-o://1538c4287c6380de159e3f885997c9c69b0bb6f06329ced48b008e731624ebbb" gracePeriod=30 Sep 17 13:22:21.157729 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.157533 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="offline" containerID="cri-o://a65231a6fc056b3abd7485779d9fd2a4de6c3f8e5a317e8553790c5f017d4478" gracePeriod=30 Sep 17 13:22:21.157729 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.157565 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" containerID="cri-o://613581b8ff19d607f88d8b5acb7c09cc5d1f20f79db4980d768cdf58a4728480" gracePeriod=30 Sep 17 13:22:21.592933 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.592891 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4"] Sep 17 13:22:21.593472 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.593442 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc1c8dda-aed1-48c8-9557-08248c0a2727" containerName="console" Sep 17 13:22:21.593472 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.593466 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc1c8dda-aed1-48c8-9557-08248c0a2727" containerName="console" Sep 17 13:22:21.593680 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.593564 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc1c8dda-aed1-48c8-9557-08248c0a2727" containerName="console" Sep 17 13:22:21.597775 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.597753 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.599541 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.599517 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-2bdvx\"" Sep 17 13:22:21.609358 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.609333 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4"] Sep 17 13:22:21.692184 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.692147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-online-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.692400 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.692197 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-feast-data\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.692400 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.692322 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-offline-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.692400 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.692359 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.692400 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.692389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.692562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.692412 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg5cd\" (UniqueName: \"kubernetes.io/projected/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-kube-api-access-rg5cd\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.793730 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.793694 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-offline-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.793895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.793744 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.793895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.793774 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.793895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.793807 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg5cd\" (UniqueName: \"kubernetes.io/projected/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-kube-api-access-rg5cd\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.793895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.793844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-online-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.793895 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:22:21.793870 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 17 13:22:21.794138 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.793878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-feast-data\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.794138 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:22:21.793928 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 17 13:22:21.794138 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:22:21.793950 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls podName:5ed0571a-8602-4c3a-a76b-a7741f2a9d9c nodeName:}" failed. No retries permitted until 2026-09-17 13:22:22.293928359 +0000 UTC m=+451.198497035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls") pod "feast-simple-feast-setup-69cd4fb9c9-vt7k4" (UID: "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c") : secret "feast-simple-feast-setup-ui-tls" not found Sep 17 13:22:21.794138 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:22:21.794018 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls podName:5ed0571a-8602-4c3a-a76b-a7741f2a9d9c nodeName:}" failed. No retries permitted until 2026-09-17 13:22:22.293994917 +0000 UTC m=+451.198563586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls") pod "feast-simple-feast-setup-69cd4fb9c9-vt7k4" (UID: "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c") : secret "feast-simple-feast-setup-registry-tls" not found Sep 17 13:22:21.794423 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.794405 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-feast-data\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.796470 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.796449 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-offline-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.796530 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.796486 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-online-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:21.810752 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:21.810725 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg5cd\" (UniqueName: \"kubernetes.io/projected/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-kube-api-access-rg5cd\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:22.299454 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.299426 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:22.299897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.299471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:22.301831 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.301808 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:22.301931 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.301839 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls\") pod \"feast-simple-feast-setup-69cd4fb9c9-vt7k4\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:22.510282 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.510249 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:22.598978 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.598949 2575 generic.go:356] "Generic (PLEG): container finished" podID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerID="1538c4287c6380de159e3f885997c9c69b0bb6f06329ced48b008e731624ebbb" exitCode=0 Sep 17 13:22:22.598978 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.598970 2575 generic.go:356] "Generic (PLEG): container finished" podID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerID="613581b8ff19d607f88d8b5acb7c09cc5d1f20f79db4980d768cdf58a4728480" exitCode=0 Sep 17 13:22:22.599136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.599022 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerDied","Data":"1538c4287c6380de159e3f885997c9c69b0bb6f06329ced48b008e731624ebbb"} Sep 17 13:22:22.599136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.599062 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerDied","Data":"613581b8ff19d607f88d8b5acb7c09cc5d1f20f79db4980d768cdf58a4728480"} Sep 17 13:22:22.789196 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:22.789173 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4"] Sep 17 13:22:22.791288 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:22:22.791240 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ed0571a_8602_4c3a_a76b_a7741f2a9d9c.slice/crio-4d1f3094f482ff7fdef1bd24d145ff857756e21d83364ad58e3b94eb2b164bd6 WatchSource:0}: Error finding container 4d1f3094f482ff7fdef1bd24d145ff857756e21d83364ad58e3b94eb2b164bd6: Status 404 returned error can't find the container with id 4d1f3094f482ff7fdef1bd24d145ff857756e21d83364ad58e3b94eb2b164bd6 Sep 17 13:22:23.606972 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:23.606934 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerStarted","Data":"754035d454829e2872facf638e3d4b06d53fc1265489ab04e22518310a02df30"} Sep 17 13:22:23.606972 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:23.606979 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerStarted","Data":"4d1f3094f482ff7fdef1bd24d145ff857756e21d83364ad58e3b94eb2b164bd6"} Sep 17 13:22:27.622552 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:27.622515 2575 generic.go:356] "Generic (PLEG): container finished" podID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerID="754035d454829e2872facf638e3d4b06d53fc1265489ab04e22518310a02df30" exitCode=0 Sep 17 13:22:27.622935 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:27.622586 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerDied","Data":"754035d454829e2872facf638e3d4b06d53fc1265489ab04e22518310a02df30"} Sep 17 13:22:28.628191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:28.628158 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerStarted","Data":"8474052c9c9f5d756883a2a497f984375b592332220a3441cb39c0adfa718479"} Sep 17 13:22:28.816240 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:28.816194 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 17 13:22:29.345166 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:29.345116 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 17 13:22:33.652058 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:33.652023 2575 generic.go:356] "Generic (PLEG): container finished" podID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerID="8474052c9c9f5d756883a2a497f984375b592332220a3441cb39c0adfa718479" exitCode=0 Sep 17 13:22:33.652474 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:33.652097 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerDied","Data":"8474052c9c9f5d756883a2a497f984375b592332220a3441cb39c0adfa718479"} Sep 17 13:22:34.663011 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:34.662968 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerStarted","Data":"c1c63460bb0d475ca6fa3f4cc3b3d0d23dce09ae787c2af581e66578f8d01a1c"} Sep 17 13:22:34.663485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:34.663038 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerStarted","Data":"569970f08b051102efecd224d1abdf3daae6ece994223033552e8d881c2f8684"} Sep 17 13:22:34.663485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:34.663059 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerStarted","Data":"039b32f0814eb85b53e78a3a17f2666eef682ae409c61efa2ae180ac5c3b7adf"} Sep 17 13:22:34.663485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:34.663077 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerStarted","Data":"095188f4c9cd3b7c873f54c2b898828adafa1ecdc78398ce564d91f69c331f17"} Sep 17 13:22:34.687460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:34.687388 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podStartSLOduration=13.687365985 podStartE2EDuration="13.687365985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:22:34.68664192 +0000 UTC m=+463.591210597" watchObservedRunningTime="2026-09-17 13:22:34.687365985 +0000 UTC m=+463.591934663" Sep 17 13:22:37.511517 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:37.511471 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:37.511967 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:37.511544 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:37.511967 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:37.511564 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:37.511967 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:37.511582 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:37.513183 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:37.513152 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Sep 17 13:22:37.513441 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:37.513179 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 17 13:22:37.513553 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:37.513199 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 17 13:22:37.513553 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:37.513217 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 17 13:22:38.816353 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:38.816309 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 17 13:22:39.345871 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:39.345824 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 17 13:22:40.510622 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:40.510578 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 17 13:22:40.511083 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:40.510578 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 17 13:22:40.511083 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:40.510578 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Sep 17 13:22:40.511264 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:40.511235 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 17 13:22:42.511247 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:42.511207 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:42.511627 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:42.511297 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:42.511627 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:42.511323 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:42.511627 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:42.511338 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:43.511209 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:43.511176 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:43.511540 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:43.511521 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:43.511857 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:43.511657 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:43.516371 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:43.516344 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:43.716174 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:43.716147 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:43.716332 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:43.716320 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:43.716386 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:43.716367 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:43.718718 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:43.718703 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:22:48.815981 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:48.815933 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Sep 17 13:22:48.816444 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:48.816073 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:22:49.345758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:49.345717 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Sep 17 13:22:49.345925 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:49.345857 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:22:51.751382 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.751344 2575 generic.go:356] "Generic (PLEG): container finished" podID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerID="a65231a6fc056b3abd7485779d9fd2a4de6c3f8e5a317e8553790c5f017d4478" exitCode=137 Sep 17 13:22:51.751382 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.751381 2575 generic.go:356] "Generic (PLEG): container finished" podID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerID="f86f3dc9c21dcee201db00711921de6139ae870aeff9fa7acec02190231ec41a" exitCode=137 Sep 17 13:22:51.751718 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.751433 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerDied","Data":"a65231a6fc056b3abd7485779d9fd2a4de6c3f8e5a317e8553790c5f017d4478"} Sep 17 13:22:51.751718 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.751483 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerDied","Data":"f86f3dc9c21dcee201db00711921de6139ae870aeff9fa7acec02190231ec41a"} Sep 17 13:22:51.795621 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.795599 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:22:51.844257 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.844230 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-registry-tls\") pod \"b9192979-77cb-4a67-9d92-fc9321a549a0\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " Sep 17 13:22:51.844431 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.844293 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b9192979-77cb-4a67-9d92-fc9321a549a0-feast-data\") pod \"b9192979-77cb-4a67-9d92-fc9321a549a0\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " Sep 17 13:22:51.844431 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.844406 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-offline-tls\") pod \"b9192979-77cb-4a67-9d92-fc9321a549a0\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " Sep 17 13:22:51.844501 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.844442 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls\") pod \"b9192979-77cb-4a67-9d92-fc9321a549a0\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " Sep 17 13:22:51.844501 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.844495 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-online-tls\") pod \"b9192979-77cb-4a67-9d92-fc9321a549a0\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " Sep 17 13:22:51.844577 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.844522 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8cwq\" (UniqueName: \"kubernetes.io/projected/b9192979-77cb-4a67-9d92-fc9321a549a0-kube-api-access-w8cwq\") pod \"b9192979-77cb-4a67-9d92-fc9321a549a0\" (UID: \"b9192979-77cb-4a67-9d92-fc9321a549a0\") " Sep 17 13:22:51.844829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.844783 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9192979-77cb-4a67-9d92-fc9321a549a0-feast-data" (OuterVolumeSpecName: "feast-data") pod "b9192979-77cb-4a67-9d92-fc9321a549a0" (UID: "b9192979-77cb-4a67-9d92-fc9321a549a0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:22:51.846740 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.846677 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-online-tls" (OuterVolumeSpecName: "online-tls") pod "b9192979-77cb-4a67-9d92-fc9321a549a0" (UID: "b9192979-77cb-4a67-9d92-fc9321a549a0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:22:51.846841 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.846742 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9192979-77cb-4a67-9d92-fc9321a549a0-kube-api-access-w8cwq" (OuterVolumeSpecName: "kube-api-access-w8cwq") pod "b9192979-77cb-4a67-9d92-fc9321a549a0" (UID: "b9192979-77cb-4a67-9d92-fc9321a549a0"). InnerVolumeSpecName "kube-api-access-w8cwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:22:51.846841 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.846769 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b9192979-77cb-4a67-9d92-fc9321a549a0" (UID: "b9192979-77cb-4a67-9d92-fc9321a549a0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:22:51.847017 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.846996 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b9192979-77cb-4a67-9d92-fc9321a549a0" (UID: "b9192979-77cb-4a67-9d92-fc9321a549a0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:22:51.847070 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.847020 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b9192979-77cb-4a67-9d92-fc9321a549a0" (UID: "b9192979-77cb-4a67-9d92-fc9321a549a0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:22:51.945753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.945695 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:22:51.945753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.945717 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b9192979-77cb-4a67-9d92-fc9321a549a0-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:22:51.945753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.945725 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:22:51.945753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.945732 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:22:51.945753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.945755 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b9192979-77cb-4a67-9d92-fc9321a549a0-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:22:51.945958 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:51.945763 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8cwq\" (UniqueName: \"kubernetes.io/projected/b9192979-77cb-4a67-9d92-fc9321a549a0-kube-api-access-w8cwq\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:22:52.758973 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.758936 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" event={"ID":"b9192979-77cb-4a67-9d92-fc9321a549a0","Type":"ContainerDied","Data":"ea8292c2a73502506907e5f0f4c6b01931a5ea0cbddaf01b5c1b185f9e40986b"} Sep 17 13:22:52.759378 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.758997 2575 scope.go:117] "RemoveContainer" containerID="1538c4287c6380de159e3f885997c9c69b0bb6f06329ced48b008e731624ebbb" Sep 17 13:22:52.759378 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.758950 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk" Sep 17 13:22:52.767735 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.767719 2575 scope.go:117] "RemoveContainer" containerID="a65231a6fc056b3abd7485779d9fd2a4de6c3f8e5a317e8553790c5f017d4478" Sep 17 13:22:52.775233 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.775215 2575 scope.go:117] "RemoveContainer" containerID="613581b8ff19d607f88d8b5acb7c09cc5d1f20f79db4980d768cdf58a4728480" Sep 17 13:22:52.782289 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.782252 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk"] Sep 17 13:22:52.783090 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.783073 2575 scope.go:117] "RemoveContainer" containerID="f86f3dc9c21dcee201db00711921de6139ae870aeff9fa7acec02190231ec41a" Sep 17 13:22:52.788666 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.788644 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-6s5fk"] Sep 17 13:22:52.791138 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.791121 2575 scope.go:117] "RemoveContainer" containerID="f89ceee1d9eb49cc980c6f28a8fe13e81de512c84980f2fa95cb2edfff405d4f" Sep 17 13:22:52.798223 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:52.798204 2575 scope.go:117] "RemoveContainer" containerID="2ce8f43b89e5ba5c3bb20647692bc349258a9a03b5254f6ef9962506b215f365" Sep 17 13:22:53.608140 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:22:53.608110 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" path="/var/lib/kubelet/pods/b9192979-77cb-4a67-9d92-fc9321a549a0/volumes" Sep 17 13:23:46.791712 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.791627 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w"] Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792089 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="feast-apply" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792106 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="feast-apply" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792117 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792125 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792135 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="registry" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792143 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="registry" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792152 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792159 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792169 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="feast-init" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792176 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="feast-init" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792199 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="offline" Sep 17 13:23:46.792235 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792206 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="offline" Sep 17 13:23:46.792851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792314 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="ui" Sep 17 13:23:46.792851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792330 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="registry" Sep 17 13:23:46.792851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792338 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="offline" Sep 17 13:23:46.792851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.792350 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9192979-77cb-4a67-9d92-fc9321a549a0" containerName="online" Sep 17 13:23:46.795658 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.795637 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:46.797744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.797723 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 13:23:46.797744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.797726 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 13:23:46.797959 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.797730 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 13:23:46.797959 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.797731 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 13:23:46.798117 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.798100 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 17 13:23:46.798214 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.798195 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 13:23:46.798347 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.798259 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 13:23:46.798422 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.798406 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-89rp9\"" Sep 17 13:23:46.807125 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.807105 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w"] Sep 17 13:23:46.992428 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.992394 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:46.992586 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.992459 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:46.992586 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.992496 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnn9f\" (UniqueName: \"kubernetes.io/projected/c69f0140-914f-4184-953a-ca89209866ec-kube-api-access-tnn9f\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:46.992586 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.992531 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c69f0140-914f-4184-953a-ca89209866ec-registry-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:46.992707 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.992584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:46.992707 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:46.992634 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c69f0140-914f-4184-953a-ca89209866ec-feast-data\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.093051 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.092972 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.093051 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.093013 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnn9f\" (UniqueName: \"kubernetes.io/projected/c69f0140-914f-4184-953a-ca89209866ec-kube-api-access-tnn9f\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.093051 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.093046 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c69f0140-914f-4184-953a-ca89209866ec-registry-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.093328 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.093075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.093328 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:23:47.093108 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 17 13:23:47.093328 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:23:47.093175 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls podName:c69f0140-914f-4184-953a-ca89209866ec nodeName:}" failed. No retries permitted until 2026-09-17 13:23:47.593159738 +0000 UTC m=+536.497728393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls") pod "feast-simple-feast-remote-setup-74fd8664d9-7wh9w" (UID: "c69f0140-914f-4184-953a-ca89209866ec") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 17 13:23:47.093328 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.093112 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c69f0140-914f-4184-953a-ca89209866ec-feast-data\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.093328 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:23:47.093195 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 13:23:47.093328 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.093237 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.093328 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:23:47.093308 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls podName:c69f0140-914f-4184-953a-ca89209866ec nodeName:}" failed. No retries permitted until 2026-09-17 13:23:47.593293397 +0000 UTC m=+536.497862065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls") pod "feast-simple-feast-remote-setup-74fd8664d9-7wh9w" (UID: "c69f0140-914f-4184-953a-ca89209866ec") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 13:23:47.093328 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:23:47.093327 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 13:23:47.093628 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:23:47.093377 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls podName:c69f0140-914f-4184-953a-ca89209866ec nodeName:}" failed. No retries permitted until 2026-09-17 13:23:47.593358281 +0000 UTC m=+536.497926956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls") pod "feast-simple-feast-remote-setup-74fd8664d9-7wh9w" (UID: "c69f0140-914f-4184-953a-ca89209866ec") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 13:23:47.093628 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.093562 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c69f0140-914f-4184-953a-ca89209866ec-feast-data\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.093743 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.093725 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c69f0140-914f-4184-953a-ca89209866ec-registry-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.104884 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.104863 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnn9f\" (UniqueName: \"kubernetes.io/projected/c69f0140-914f-4184-953a-ca89209866ec-kube-api-access-tnn9f\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.598053 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.598023 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.598230 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.598076 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.598230 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.598114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.598230 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:23:47.598213 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 13:23:47.598387 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:23:47.598308 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls podName:c69f0140-914f-4184-953a-ca89209866ec nodeName:}" failed. No retries permitted until 2026-09-17 13:23:48.598293597 +0000 UTC m=+537.502862266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls") pod "feast-simple-feast-remote-setup-74fd8664d9-7wh9w" (UID: "c69f0140-914f-4184-953a-ca89209866ec") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 13:23:47.600555 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.600523 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:47.600687 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:47.600643 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:48.606988 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:48.606954 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:48.609416 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:48.609395 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls\") pod \"feast-simple-feast-remote-setup-74fd8664d9-7wh9w\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:48.904674 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:48.904597 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:23:49.182797 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:49.182774 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w"] Sep 17 13:23:49.184959 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:23:49.184933 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc69f0140_914f_4184_953a_ca89209866ec.slice/crio-dd0519c98506a74885bd00d4ed8f68e1315400e11da74b384c4cd8cbfe1808ac WatchSource:0}: Error finding container dd0519c98506a74885bd00d4ed8f68e1315400e11da74b384c4cd8cbfe1808ac: Status 404 returned error can't find the container with id dd0519c98506a74885bd00d4ed8f68e1315400e11da74b384c4cd8cbfe1808ac Sep 17 13:23:49.983800 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:49.983758 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerStarted","Data":"44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59"} Sep 17 13:23:49.983800 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:49.983801 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerStarted","Data":"dd0519c98506a74885bd00d4ed8f68e1315400e11da74b384c4cd8cbfe1808ac"} Sep 17 13:23:54.006987 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:54.006948 2575 generic.go:356] "Generic (PLEG): container finished" podID="c69f0140-914f-4184-953a-ca89209866ec" containerID="44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59" exitCode=0 Sep 17 13:23:54.007374 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:54.007004 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerDied","Data":"44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59"} Sep 17 13:23:55.013414 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:23:55.013367 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerStarted","Data":"b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2"} Sep 17 13:24:20.113098 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:20.113062 2575 generic.go:356] "Generic (PLEG): container finished" podID="c69f0140-914f-4184-953a-ca89209866ec" containerID="b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2" exitCode=0 Sep 17 13:24:20.113612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:20.113105 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerDied","Data":"b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2"} Sep 17 13:24:20.115953 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:20.115933 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 17 13:24:21.121724 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.121689 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerStarted","Data":"075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142"} Sep 17 13:24:21.122220 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.121747 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerStarted","Data":"abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234"} Sep 17 13:24:21.122220 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.121773 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerStarted","Data":"8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1"} Sep 17 13:24:21.142463 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.142400 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podStartSLOduration=35.142381511 podStartE2EDuration="35.142381511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:23:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:24:21.141484419 +0000 UTC m=+570.046053136" watchObservedRunningTime="2026-09-17 13:24:21.142381511 +0000 UTC m=+570.046950189" Sep 17 13:24:21.905153 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.905113 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:21.905363 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.905182 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:21.905363 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.905200 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:21.906974 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.906936 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 17 13:24:21.907116 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.906977 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 17 13:24:21.907116 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:21.906988 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 17 13:24:24.905807 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:24.905758 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 17 13:24:24.906307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:24.905817 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 17 13:24:24.906307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:24.905767 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 17 13:24:27.904927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:27.904874 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 17 13:24:27.905493 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:27.905474 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:27.905895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:27.905871 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:27.906313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:27.906291 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:27.927997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:27.927969 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:28.160989 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:28.160906 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:28.181436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:28.181411 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:28.905200 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:28.905162 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:30.905878 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:30.905844 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:31.173647 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:31.173576 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:24:51.511195 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:51.511169 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:24:51.513797 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:24:51.513778 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:25:33.056736 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.056701 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w"] Sep 17 13:25:33.057191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.056957 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" containerID="cri-o://8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1" gracePeriod=30 Sep 17 13:25:33.057191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.057018 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" containerID="cri-o://075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142" gracePeriod=30 Sep 17 13:25:33.057191 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.057025 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="offline" containerID="cri-o://abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234" gracePeriod=30 Sep 17 13:25:33.158503 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.158471 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4"] Sep 17 13:25:33.158786 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.158754 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="registry" containerID="cri-o://095188f4c9cd3b7c873f54c2b898828adafa1ecdc78398ce564d91f69c331f17" gracePeriod=30 Sep 17 13:25:33.158890 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.158825 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="offline" containerID="cri-o://569970f08b051102efecd224d1abdf3daae6ece994223033552e8d881c2f8684" gracePeriod=30 Sep 17 13:25:33.158890 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.158863 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" containerID="cri-o://039b32f0814eb85b53e78a3a17f2666eef682ae409c61efa2ae180ac5c3b7adf" gracePeriod=30 Sep 17 13:25:33.158995 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.158959 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" containerID="cri-o://c1c63460bb0d475ca6fa3f4cc3b3d0d23dce09ae787c2af581e66578f8d01a1c" gracePeriod=30 Sep 17 13:25:33.716492 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.716432 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 17 13:25:33.716831 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:33.716802 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 17 13:25:34.434229 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:34.434200 2575 generic.go:356] "Generic (PLEG): container finished" podID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerID="c1c63460bb0d475ca6fa3f4cc3b3d0d23dce09ae787c2af581e66578f8d01a1c" exitCode=0 Sep 17 13:25:34.434229 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:34.434222 2575 generic.go:356] "Generic (PLEG): container finished" podID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerID="039b32f0814eb85b53e78a3a17f2666eef682ae409c61efa2ae180ac5c3b7adf" exitCode=0 Sep 17 13:25:34.434730 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:34.434287 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerDied","Data":"c1c63460bb0d475ca6fa3f4cc3b3d0d23dce09ae787c2af581e66578f8d01a1c"} Sep 17 13:25:34.434730 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:34.434328 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerDied","Data":"039b32f0814eb85b53e78a3a17f2666eef682ae409c61efa2ae180ac5c3b7adf"} Sep 17 13:25:34.436885 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:34.436865 2575 generic.go:356] "Generic (PLEG): container finished" podID="c69f0140-914f-4184-953a-ca89209866ec" containerID="075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142" exitCode=0 Sep 17 13:25:34.436885 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:34.436882 2575 generic.go:356] "Generic (PLEG): container finished" podID="c69f0140-914f-4184-953a-ca89209866ec" containerID="8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1" exitCode=0 Sep 17 13:25:34.437023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:34.436931 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerDied","Data":"075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142"} Sep 17 13:25:34.437023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:34.436960 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerDied","Data":"8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1"} Sep 17 13:25:38.161664 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:38.161632 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 17 13:25:41.173639 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:41.173602 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 17 13:25:43.716579 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:43.716519 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 17 13:25:43.716579 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:43.716541 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 17 13:25:48.161321 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:48.161258 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 17 13:25:51.173474 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:51.173436 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 17 13:25:53.716132 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:53.716082 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 17 13:25:53.716634 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:53.716085 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 17 13:25:53.716634 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:53.716234 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:25:53.716634 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:53.716350 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:25:58.161129 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:58.161088 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 17 13:25:58.161533 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:25:58.161199 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:26:01.173355 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:01.173326 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 17 13:26:01.173749 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:01.173428 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:26:03.263130 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.263109 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:26:03.327833 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.327765 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls\") pod \"c69f0140-914f-4184-953a-ca89209866ec\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " Sep 17 13:26:03.327833 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.327809 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c69f0140-914f-4184-953a-ca89209866ec-registry-tls\") pod \"c69f0140-914f-4184-953a-ca89209866ec\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " Sep 17 13:26:03.328023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.327851 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls\") pod \"c69f0140-914f-4184-953a-ca89209866ec\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " Sep 17 13:26:03.328023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.327866 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c69f0140-914f-4184-953a-ca89209866ec-feast-data\") pod \"c69f0140-914f-4184-953a-ca89209866ec\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " Sep 17 13:26:03.328023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.327884 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnn9f\" (UniqueName: \"kubernetes.io/projected/c69f0140-914f-4184-953a-ca89209866ec-kube-api-access-tnn9f\") pod \"c69f0140-914f-4184-953a-ca89209866ec\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " Sep 17 13:26:03.328023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.328000 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls\") pod \"c69f0140-914f-4184-953a-ca89209866ec\" (UID: \"c69f0140-914f-4184-953a-ca89209866ec\") " Sep 17 13:26:03.328265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.328243 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69f0140-914f-4184-953a-ca89209866ec-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c69f0140-914f-4184-953a-ca89209866ec" (UID: "c69f0140-914f-4184-953a-ca89209866ec"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:26:03.328398 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.328377 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c69f0140-914f-4184-953a-ca89209866ec-feast-data" (OuterVolumeSpecName: "feast-data") pod "c69f0140-914f-4184-953a-ca89209866ec" (UID: "c69f0140-914f-4184-953a-ca89209866ec"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:26:03.330140 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.330114 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c69f0140-914f-4184-953a-ca89209866ec" (UID: "c69f0140-914f-4184-953a-ca89209866ec"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:26:03.330253 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.330172 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls" (OuterVolumeSpecName: "online-tls") pod "c69f0140-914f-4184-953a-ca89209866ec" (UID: "c69f0140-914f-4184-953a-ca89209866ec"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:26:03.330253 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.330182 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c69f0140-914f-4184-953a-ca89209866ec" (UID: "c69f0140-914f-4184-953a-ca89209866ec"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:26:03.330253 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.330232 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c69f0140-914f-4184-953a-ca89209866ec-kube-api-access-tnn9f" (OuterVolumeSpecName: "kube-api-access-tnn9f") pod "c69f0140-914f-4184-953a-ca89209866ec" (UID: "c69f0140-914f-4184-953a-ca89209866ec"). InnerVolumeSpecName "kube-api-access-tnn9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:26:03.429404 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.429370 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:03.429404 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.429398 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c69f0140-914f-4184-953a-ca89209866ec-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:03.429404 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.429409 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnn9f\" (UniqueName: \"kubernetes.io/projected/c69f0140-914f-4184-953a-ca89209866ec-kube-api-access-tnn9f\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:03.429621 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.429418 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:03.429621 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.429426 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c69f0140-914f-4184-953a-ca89209866ec-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:03.429621 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.429434 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c69f0140-914f-4184-953a-ca89209866ec-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:03.562907 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.562873 2575 generic.go:356] "Generic (PLEG): container finished" podID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerID="569970f08b051102efecd224d1abdf3daae6ece994223033552e8d881c2f8684" exitCode=137 Sep 17 13:26:03.562907 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.562894 2575 generic.go:356] "Generic (PLEG): container finished" podID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerID="095188f4c9cd3b7c873f54c2b898828adafa1ecdc78398ce564d91f69c331f17" exitCode=137 Sep 17 13:26:03.563110 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.562944 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerDied","Data":"569970f08b051102efecd224d1abdf3daae6ece994223033552e8d881c2f8684"} Sep 17 13:26:03.563110 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.562988 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerDied","Data":"095188f4c9cd3b7c873f54c2b898828adafa1ecdc78398ce564d91f69c331f17"} Sep 17 13:26:03.565233 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.565213 2575 generic.go:356] "Generic (PLEG): container finished" podID="c69f0140-914f-4184-953a-ca89209866ec" containerID="abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234" exitCode=137 Sep 17 13:26:03.565372 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.565258 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerDied","Data":"abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234"} Sep 17 13:26:03.565372 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.565285 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" Sep 17 13:26:03.565372 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.565320 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w" event={"ID":"c69f0140-914f-4184-953a-ca89209866ec","Type":"ContainerDied","Data":"dd0519c98506a74885bd00d4ed8f68e1315400e11da74b384c4cd8cbfe1808ac"} Sep 17 13:26:03.565372 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.565346 2575 scope.go:117] "RemoveContainer" containerID="075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142" Sep 17 13:26:03.573827 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.573812 2575 scope.go:117] "RemoveContainer" containerID="abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234" Sep 17 13:26:03.580893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.580877 2575 scope.go:117] "RemoveContainer" containerID="8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1" Sep 17 13:26:03.591478 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.591457 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w"] Sep 17 13:26:03.594025 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.594007 2575 scope.go:117] "RemoveContainer" containerID="b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2" Sep 17 13:26:03.594182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.594154 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-74fd8664d9-7wh9w"] Sep 17 13:26:03.601006 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.600984 2575 scope.go:117] "RemoveContainer" containerID="44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59" Sep 17 13:26:03.607592 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.607569 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c69f0140-914f-4184-953a-ca89209866ec" path="/var/lib/kubelet/pods/c69f0140-914f-4184-953a-ca89209866ec/volumes" Sep 17 13:26:03.608314 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.608297 2575 scope.go:117] "RemoveContainer" containerID="075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142" Sep 17 13:26:03.608555 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:26:03.608536 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142\": container with ID starting with 075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142 not found: ID does not exist" containerID="075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142" Sep 17 13:26:03.608608 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.608562 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142"} err="failed to get container status \"075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142\": rpc error: code = NotFound desc = could not find container \"075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142\": container with ID starting with 075d0af221b8f2961effb20aaf361a7d211999d4fa75100b60827ec8fb3cf142 not found: ID does not exist" Sep 17 13:26:03.608608 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.608579 2575 scope.go:117] "RemoveContainer" containerID="abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234" Sep 17 13:26:03.608801 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:26:03.608785 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234\": container with ID starting with abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234 not found: ID does not exist" containerID="abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234" Sep 17 13:26:03.608842 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.608808 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234"} err="failed to get container status \"abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234\": rpc error: code = NotFound desc = could not find container \"abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234\": container with ID starting with abd8e0a1a9905dff638015cf73ccdb24c3cec593dd3f7271ff4e85b0b78df234 not found: ID does not exist" Sep 17 13:26:03.608842 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.608823 2575 scope.go:117] "RemoveContainer" containerID="8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1" Sep 17 13:26:03.609062 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:26:03.609032 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1\": container with ID starting with 8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1 not found: ID does not exist" containerID="8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1" Sep 17 13:26:03.609139 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.609061 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1"} err="failed to get container status \"8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1\": rpc error: code = NotFound desc = could not find container \"8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1\": container with ID starting with 8fc73f14b5b89359fc43b980a7a9d276d82bccfb002a9e784afb50284df7c0c1 not found: ID does not exist" Sep 17 13:26:03.609139 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.609081 2575 scope.go:117] "RemoveContainer" containerID="b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2" Sep 17 13:26:03.609387 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:26:03.609369 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2\": container with ID starting with b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2 not found: ID does not exist" containerID="b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2" Sep 17 13:26:03.609448 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.609395 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2"} err="failed to get container status \"b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2\": rpc error: code = NotFound desc = could not find container \"b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2\": container with ID starting with b84d0693483e5f753be2f5d1c4c715498ad420d67ff7442027a6e792bb1997b2 not found: ID does not exist" Sep 17 13:26:03.609448 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.609410 2575 scope.go:117] "RemoveContainer" containerID="44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59" Sep 17 13:26:03.609640 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:26:03.609625 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59\": container with ID starting with 44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59 not found: ID does not exist" containerID="44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59" Sep 17 13:26:03.609677 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.609642 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59"} err="failed to get container status \"44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59\": rpc error: code = NotFound desc = could not find container \"44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59\": container with ID starting with 44037c26905f6c928ca19ac1410f9135985ac1addeddc835ed038f8801e85d59 not found: ID does not exist" Sep 17 13:26:03.716724 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.716690 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 17 13:26:03.716876 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.716693 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Sep 17 13:26:03.716876 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.716697 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 17 13:26:03.716876 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.716697 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 17 13:26:03.794364 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.794342 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:26:03.933327 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.933244 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-offline-tls\") pod \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " Sep 17 13:26:03.933327 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.933289 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls\") pod \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " Sep 17 13:26:03.933501 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.933332 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls\") pod \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " Sep 17 13:26:03.933575 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.933559 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-online-tls\") pod \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " Sep 17 13:26:03.933627 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.933602 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-feast-data\") pod \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " Sep 17 13:26:03.933679 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.933629 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg5cd\" (UniqueName: \"kubernetes.io/projected/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-kube-api-access-rg5cd\") pod \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\" (UID: \"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c\") " Sep 17 13:26:03.934427 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.934395 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-feast-data" (OuterVolumeSpecName: "feast-data") pod "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" (UID: "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:26:03.935461 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.935432 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" (UID: "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:26:03.935553 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.935516 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" (UID: "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:26:03.935594 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.935565 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-online-tls" (OuterVolumeSpecName: "online-tls") pod "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" (UID: "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:26:03.935948 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.935935 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" (UID: "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:26:03.936089 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:03.936074 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-kube-api-access-rg5cd" (OuterVolumeSpecName: "kube-api-access-rg5cd") pod "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" (UID: "5ed0571a-8602-4c3a-a76b-a7741f2a9d9c"). InnerVolumeSpecName "kube-api-access-rg5cd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:26:04.034229 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.034203 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:04.034229 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.034225 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:04.034390 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.034234 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg5cd\" (UniqueName: \"kubernetes.io/projected/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-kube-api-access-rg5cd\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:04.034390 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.034243 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:04.034390 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.034252 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:04.034390 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.034260 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:26:04.572780 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.572743 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" event={"ID":"5ed0571a-8602-4c3a-a76b-a7741f2a9d9c","Type":"ContainerDied","Data":"4d1f3094f482ff7fdef1bd24d145ff857756e21d83364ad58e3b94eb2b164bd6"} Sep 17 13:26:04.572780 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.572769 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4" Sep 17 13:26:04.573200 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.572789 2575 scope.go:117] "RemoveContainer" containerID="c1c63460bb0d475ca6fa3f4cc3b3d0d23dce09ae787c2af581e66578f8d01a1c" Sep 17 13:26:04.581727 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.581705 2575 scope.go:117] "RemoveContainer" containerID="569970f08b051102efecd224d1abdf3daae6ece994223033552e8d881c2f8684" Sep 17 13:26:04.590609 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.590574 2575 scope.go:117] "RemoveContainer" containerID="039b32f0814eb85b53e78a3a17f2666eef682ae409c61efa2ae180ac5c3b7adf" Sep 17 13:26:04.596254 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.596230 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4"] Sep 17 13:26:04.598700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.598681 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-69cd4fb9c9-vt7k4"] Sep 17 13:26:04.599974 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.599959 2575 scope.go:117] "RemoveContainer" containerID="095188f4c9cd3b7c873f54c2b898828adafa1ecdc78398ce564d91f69c331f17" Sep 17 13:26:04.607518 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.607500 2575 scope.go:117] "RemoveContainer" containerID="8474052c9c9f5d756883a2a497f984375b592332220a3441cb39c0adfa718479" Sep 17 13:26:04.614145 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:04.614126 2575 scope.go:117] "RemoveContainer" containerID="754035d454829e2872facf638e3d4b06d53fc1265489ab04e22518310a02df30" Sep 17 13:26:05.607022 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:05.606993 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" path="/var/lib/kubelet/pods/5ed0571a-8602-4c3a-a76b-a7741f2a9d9c/volumes" Sep 17 13:26:16.202373 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202330 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-ck26p"] Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202623 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202633 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202643 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="registry" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202649 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="registry" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202654 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202660 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202667 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="feast-apply" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202671 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="feast-apply" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202685 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="feast-init" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202690 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="feast-init" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202701 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="offline" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202706 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="offline" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202712 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="offline" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202717 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="offline" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202725 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202729 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202734 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="feast-init" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202738 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="feast-init" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202744 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="feast-apply" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202748 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="feast-apply" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202753 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202757 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202800 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="offline" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202808 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="online" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202814 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="ui" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202819 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="offline" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202825 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="ui" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202829 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed0571a-8602-4c3a-a76b-a7741f2a9d9c" containerName="registry" Sep 17 13:26:16.202844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.202836 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69f0140-914f-4184-953a-ca89209866ec" containerName="online" Sep 17 13:26:16.205520 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.205502 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:26:16.207309 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.207284 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 13:26:16.207426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.207409 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 13:26:16.207468 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.207435 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 17 13:26:16.207897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.207879 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-jvj7l\"" Sep 17 13:26:16.214424 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.214405 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-ck26p"] Sep 17 13:26:16.239641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.239620 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-62rx2"] Sep 17 13:26:16.242652 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.242633 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-62rx2" Sep 17 13:26:16.250846 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.250827 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-62rx2"] Sep 17 13:26:16.329618 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.329580 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/54c1c6b5-310f-4e92-9947-c9e4c9768991-postgresdata\") pod \"postgres-864846d4f5-ck26p\" (UID: \"54c1c6b5-310f-4e92-9947-c9e4c9768991\") " pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:26:16.329761 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.329645 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78rtg\" (UniqueName: \"kubernetes.io/projected/c2324eda-2c55-402f-938d-9476b42a47d1-kube-api-access-78rtg\") pod \"redis-69bbd798b6-62rx2\" (UID: \"c2324eda-2c55-402f-938d-9476b42a47d1\") " pod="test-ns-feast/redis-69bbd798b6-62rx2" Sep 17 13:26:16.329761 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.329676 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s969x\" (UniqueName: \"kubernetes.io/projected/54c1c6b5-310f-4e92-9947-c9e4c9768991-kube-api-access-s969x\") pod \"postgres-864846d4f5-ck26p\" (UID: \"54c1c6b5-310f-4e92-9947-c9e4c9768991\") " pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:26:16.430609 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.430577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/54c1c6b5-310f-4e92-9947-c9e4c9768991-postgresdata\") pod \"postgres-864846d4f5-ck26p\" (UID: \"54c1c6b5-310f-4e92-9947-c9e4c9768991\") " pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:26:16.430609 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.430612 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78rtg\" (UniqueName: \"kubernetes.io/projected/c2324eda-2c55-402f-938d-9476b42a47d1-kube-api-access-78rtg\") pod \"redis-69bbd798b6-62rx2\" (UID: \"c2324eda-2c55-402f-938d-9476b42a47d1\") " pod="test-ns-feast/redis-69bbd798b6-62rx2" Sep 17 13:26:16.430793 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.430633 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s969x\" (UniqueName: \"kubernetes.io/projected/54c1c6b5-310f-4e92-9947-c9e4c9768991-kube-api-access-s969x\") pod \"postgres-864846d4f5-ck26p\" (UID: \"54c1c6b5-310f-4e92-9947-c9e4c9768991\") " pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:26:16.430976 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.430959 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/54c1c6b5-310f-4e92-9947-c9e4c9768991-postgresdata\") pod \"postgres-864846d4f5-ck26p\" (UID: \"54c1c6b5-310f-4e92-9947-c9e4c9768991\") " pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:26:16.440136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.440103 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s969x\" (UniqueName: \"kubernetes.io/projected/54c1c6b5-310f-4e92-9947-c9e4c9768991-kube-api-access-s969x\") pod \"postgres-864846d4f5-ck26p\" (UID: \"54c1c6b5-310f-4e92-9947-c9e4c9768991\") " pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:26:16.440136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.440113 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78rtg\" (UniqueName: \"kubernetes.io/projected/c2324eda-2c55-402f-938d-9476b42a47d1-kube-api-access-78rtg\") pod \"redis-69bbd798b6-62rx2\" (UID: \"c2324eda-2c55-402f-938d-9476b42a47d1\") " pod="test-ns-feast/redis-69bbd798b6-62rx2" Sep 17 13:26:16.525000 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.524975 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:26:16.551958 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.551924 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-62rx2" Sep 17 13:26:16.795037 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.794956 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-ck26p"] Sep 17 13:26:16.798266 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:26:16.798244 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54c1c6b5_310f_4e92_9947_c9e4c9768991.slice/crio-0b7be6048940547f071265a6910c96621e7f88753b952546524c50540900fe60 WatchSource:0}: Error finding container 0b7be6048940547f071265a6910c96621e7f88753b952546524c50540900fe60: Status 404 returned error can't find the container with id 0b7be6048940547f071265a6910c96621e7f88753b952546524c50540900fe60 Sep 17 13:26:16.800000 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.799983 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 13:26:16.822876 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:16.822854 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-62rx2"] Sep 17 13:26:16.825232 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:26:16.825214 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2324eda_2c55_402f_938d_9476b42a47d1.slice/crio-89c89c46378189147b5305a2dbd25710e919af4785c9b41bbd4939027c33fa1d WatchSource:0}: Error finding container 89c89c46378189147b5305a2dbd25710e919af4785c9b41bbd4939027c33fa1d: Status 404 returned error can't find the container with id 89c89c46378189147b5305a2dbd25710e919af4785c9b41bbd4939027c33fa1d Sep 17 13:26:17.624982 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:17.624946 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-62rx2" event={"ID":"c2324eda-2c55-402f-938d-9476b42a47d1","Type":"ContainerStarted","Data":"89c89c46378189147b5305a2dbd25710e919af4785c9b41bbd4939027c33fa1d"} Sep 17 13:26:17.626546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:17.626511 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-ck26p" event={"ID":"54c1c6b5-310f-4e92-9947-c9e4c9768991","Type":"ContainerStarted","Data":"0b7be6048940547f071265a6910c96621e7f88753b952546524c50540900fe60"} Sep 17 13:26:18.632538 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:18.632501 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-62rx2" event={"ID":"c2324eda-2c55-402f-938d-9476b42a47d1","Type":"ContainerStarted","Data":"8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5"} Sep 17 13:26:18.647826 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:18.647706 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-62rx2" podStartSLOduration=1.006570479 podStartE2EDuration="2.647686531s" totalImagesPullingTime="1.641116052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:26:18.647187772 +0000 UTC m=+687.551756449" watchObservedRunningTime="2026-09-17 13:26:18.647686531 +0000 UTC m=+687.552255207" Sep 17 13:26:21.645630 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:21.645594 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-ck26p" event={"ID":"54c1c6b5-310f-4e92-9947-c9e4c9768991","Type":"ContainerStarted","Data":"030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3"} Sep 17 13:26:21.661860 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:21.661801 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-ck26p" podStartSLOduration=1.560294456 podStartE2EDuration="5.66178038s" totalImagesPullingTime="4.101485924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:26:21.660539232 +0000 UTC m=+690.565107908" watchObservedRunningTime="2026-09-17 13:26:21.66178038 +0000 UTC m=+690.566349059" Sep 17 13:26:25.115117 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.115081 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc"] Sep 17 13:26:25.122892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.122872 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.124906 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.124887 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 17 13:26:25.125139 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.125124 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-plh49\"" Sep 17 13:26:25.125211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.125142 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 17 13:26:25.125211 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.125152 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 17 13:26:25.125449 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.125433 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 17 13:26:25.131007 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.130986 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc"] Sep 17 13:26:25.313926 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.313891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-online-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.313926 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.313926 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2gxz\" (UniqueName: \"kubernetes.io/projected/b2d0e828-f962-47be-8330-8807472e4739-kube-api-access-j2gxz\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.314129 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.313952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-registry-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.314129 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.314044 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-offline-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.314129 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.314099 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d0e828-f962-47be-8330-8807472e4739-feast-data\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.415249 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.415175 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-online-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.415249 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.415210 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2gxz\" (UniqueName: \"kubernetes.io/projected/b2d0e828-f962-47be-8330-8807472e4739-kube-api-access-j2gxz\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.415443 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.415309 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-registry-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.415443 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.415389 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-offline-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.415521 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.415464 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d0e828-f962-47be-8330-8807472e4739-feast-data\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.415811 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.415782 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d0e828-f962-47be-8330-8807472e4739-feast-data\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.417691 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.417671 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-offline-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.417691 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.417685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-online-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.417816 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.417793 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-registry-tls\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.426122 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.426100 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2gxz\" (UniqueName: \"kubernetes.io/projected/b2d0e828-f962-47be-8330-8807472e4739-kube-api-access-j2gxz\") pod \"feast-credit-scoring-544b86bc6-5h9pc\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.432865 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.432835 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:25.709417 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:25.709351 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc"] Sep 17 13:26:25.712099 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:26:25.712075 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d0e828_f962_47be_8330_8807472e4739.slice/crio-a5b198323b822c0d451d268881582063e31aa5f01309af570a70d024de59e550 WatchSource:0}: Error finding container a5b198323b822c0d451d268881582063e31aa5f01309af570a70d024de59e550: Status 404 returned error can't find the container with id a5b198323b822c0d451d268881582063e31aa5f01309af570a70d024de59e550 Sep 17 13:26:26.665370 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:26.665336 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerStarted","Data":"41bdbd05bae3a96912a52eed6dc17dedacc8b085d64428b45213b071c71f11d5"} Sep 17 13:26:26.665734 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:26.665398 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerStarted","Data":"a5b198323b822c0d451d268881582063e31aa5f01309af570a70d024de59e550"} Sep 17 13:26:27.670847 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:27.670813 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2d0e828-f962-47be-8330-8807472e4739" containerID="41bdbd05bae3a96912a52eed6dc17dedacc8b085d64428b45213b071c71f11d5" exitCode=0 Sep 17 13:26:27.671218 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:27.670906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerDied","Data":"41bdbd05bae3a96912a52eed6dc17dedacc8b085d64428b45213b071c71f11d5"} Sep 17 13:26:28.676426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:28.676394 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerStarted","Data":"a7618b78863893da0b44159b2a3a67e50621ad6a15f14d7606f002beda1dfd98"} Sep 17 13:26:34.703685 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:34.703653 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2d0e828-f962-47be-8330-8807472e4739" containerID="a7618b78863893da0b44159b2a3a67e50621ad6a15f14d7606f002beda1dfd98" exitCode=0 Sep 17 13:26:34.704071 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:34.703700 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerDied","Data":"a7618b78863893da0b44159b2a3a67e50621ad6a15f14d7606f002beda1dfd98"} Sep 17 13:26:35.712229 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:35.712197 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerStarted","Data":"359500453df1f7206f9d4c9752a1bf60bbc060d744830e6251ec53b1c74deb49"} Sep 17 13:26:35.712643 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:35.712262 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerStarted","Data":"33d317a6ae9834e13322d1aa96f3969f54918680a3f5297b55cb7bdd834aa16a"} Sep 17 13:26:35.712643 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:35.712302 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerStarted","Data":"9cdddccf9e523f0ef70b37b51ad0588261698fe46ccc001af3c27916f5c3084d"} Sep 17 13:26:35.735669 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:35.735539 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podStartSLOduration=10.735518721 podStartE2EDuration="10.735518721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:26:35.734041381 +0000 UTC m=+704.638610111" watchObservedRunningTime="2026-09-17 13:26:35.735518721 +0000 UTC m=+704.640087398" Sep 17 13:26:37.433161 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:37.433114 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:37.433641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:37.433200 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:37.433641 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:37.433226 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:37.434637 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:37.434601 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused" Sep 17 13:26:37.434772 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:37.434592 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 17 13:26:37.434772 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:37.434618 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 17 13:26:40.433612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:40.433567 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 17 13:26:40.434182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:40.433583 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 17 13:26:40.434406 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:40.433589 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused" Sep 17 13:26:43.434052 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:43.433956 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 17 13:26:43.435006 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:43.434983 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:43.435510 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:43.435492 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:43.435940 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:43.435921 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:43.440491 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:43.440470 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:43.758653 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:43.758629 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:43.761995 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:43.761962 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:45.433386 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:45.433347 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:46.433666 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:46.433622 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 17 13:26:49.434531 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:49.434492 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:26:49.435359 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:26:49.435339 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:27:52.941752 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:52.941722 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc"] Sep 17 13:27:52.942311 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:52.942009 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="registry" containerID="cri-o://9cdddccf9e523f0ef70b37b51ad0588261698fe46ccc001af3c27916f5c3084d" gracePeriod=30 Sep 17 13:27:52.942311 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:52.942078 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" containerID="cri-o://33d317a6ae9834e13322d1aa96f3969f54918680a3f5297b55cb7bdd834aa16a" gracePeriod=30 Sep 17 13:27:52.942311 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:52.942093 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="offline" containerID="cri-o://359500453df1f7206f9d4c9752a1bf60bbc060d744830e6251ec53b1c74deb49" gracePeriod=30 Sep 17 13:27:52.960789 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:52.960763 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-ck26p"] Sep 17 13:27:52.960928 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:52.960911 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-ck26p" podUID="54c1c6b5-310f-4e92-9947-c9e4c9768991" containerName="postgres" containerID="cri-o://030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3" gracePeriod=30 Sep 17 13:27:52.970124 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:52.970090 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-62rx2"] Sep 17 13:27:52.970265 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:52.970246 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-62rx2" podUID="c2324eda-2c55-402f-938d-9476b42a47d1" containerName="redis" containerID="cri-o://8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5" gracePeriod=30 Sep 17 13:27:53.221469 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.221432 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:27:53.224285 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.224246 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-62rx2" Sep 17 13:27:53.366797 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.366767 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/54c1c6b5-310f-4e92-9947-c9e4c9768991-postgresdata\") pod \"54c1c6b5-310f-4e92-9947-c9e4c9768991\" (UID: \"54c1c6b5-310f-4e92-9947-c9e4c9768991\") " Sep 17 13:27:53.366975 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.366807 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78rtg\" (UniqueName: \"kubernetes.io/projected/c2324eda-2c55-402f-938d-9476b42a47d1-kube-api-access-78rtg\") pod \"c2324eda-2c55-402f-938d-9476b42a47d1\" (UID: \"c2324eda-2c55-402f-938d-9476b42a47d1\") " Sep 17 13:27:53.366975 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.366920 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s969x\" (UniqueName: \"kubernetes.io/projected/54c1c6b5-310f-4e92-9947-c9e4c9768991-kube-api-access-s969x\") pod \"54c1c6b5-310f-4e92-9947-c9e4c9768991\" (UID: \"54c1c6b5-310f-4e92-9947-c9e4c9768991\") " Sep 17 13:27:53.369224 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.369186 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2324eda-2c55-402f-938d-9476b42a47d1-kube-api-access-78rtg" (OuterVolumeSpecName: "kube-api-access-78rtg") pod "c2324eda-2c55-402f-938d-9476b42a47d1" (UID: "c2324eda-2c55-402f-938d-9476b42a47d1"). InnerVolumeSpecName "kube-api-access-78rtg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:27:53.369649 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.369619 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c1c6b5-310f-4e92-9947-c9e4c9768991-kube-api-access-s969x" (OuterVolumeSpecName: "kube-api-access-s969x") pod "54c1c6b5-310f-4e92-9947-c9e4c9768991" (UID: "54c1c6b5-310f-4e92-9947-c9e4c9768991"). InnerVolumeSpecName "kube-api-access-s969x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:27:53.395964 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.395924 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c1c6b5-310f-4e92-9947-c9e4c9768991-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "54c1c6b5-310f-4e92-9947-c9e4c9768991" (UID: "54c1c6b5-310f-4e92-9947-c9e4c9768991"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:27:53.468166 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.468093 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s969x\" (UniqueName: \"kubernetes.io/projected/54c1c6b5-310f-4e92-9947-c9e4c9768991-kube-api-access-s969x\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:27:53.468166 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.468120 2575 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/54c1c6b5-310f-4e92-9947-c9e4c9768991-postgresdata\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:27:53.468166 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.468129 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78rtg\" (UniqueName: \"kubernetes.io/projected/c2324eda-2c55-402f-938d-9476b42a47d1-kube-api-access-78rtg\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:27:53.758769 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:53.758736 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 17 13:27:54.063616 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.063589 2575 generic.go:356] "Generic (PLEG): container finished" podID="c2324eda-2c55-402f-938d-9476b42a47d1" containerID="8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5" exitCode=0 Sep 17 13:27:54.063980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.063627 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-62rx2" Sep 17 13:27:54.063980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.063656 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-62rx2" event={"ID":"c2324eda-2c55-402f-938d-9476b42a47d1","Type":"ContainerDied","Data":"8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5"} Sep 17 13:27:54.063980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.063686 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-62rx2" event={"ID":"c2324eda-2c55-402f-938d-9476b42a47d1","Type":"ContainerDied","Data":"89c89c46378189147b5305a2dbd25710e919af4785c9b41bbd4939027c33fa1d"} Sep 17 13:27:54.063980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.063706 2575 scope.go:117] "RemoveContainer" containerID="8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5" Sep 17 13:27:54.066769 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.066747 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2d0e828-f962-47be-8330-8807472e4739" containerID="33d317a6ae9834e13322d1aa96f3969f54918680a3f5297b55cb7bdd834aa16a" exitCode=0 Sep 17 13:27:54.066873 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.066824 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerDied","Data":"33d317a6ae9834e13322d1aa96f3969f54918680a3f5297b55cb7bdd834aa16a"} Sep 17 13:27:54.068107 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.068089 2575 generic.go:356] "Generic (PLEG): container finished" podID="54c1c6b5-310f-4e92-9947-c9e4c9768991" containerID="030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3" exitCode=0 Sep 17 13:27:54.068215 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.068123 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-ck26p" event={"ID":"54c1c6b5-310f-4e92-9947-c9e4c9768991","Type":"ContainerDied","Data":"030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3"} Sep 17 13:27:54.068215 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.068132 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-ck26p" Sep 17 13:27:54.068215 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.068158 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-ck26p" event={"ID":"54c1c6b5-310f-4e92-9947-c9e4c9768991","Type":"ContainerDied","Data":"0b7be6048940547f071265a6910c96621e7f88753b952546524c50540900fe60"} Sep 17 13:27:54.071980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.071964 2575 scope.go:117] "RemoveContainer" containerID="8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5" Sep 17 13:27:54.072237 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:27:54.072218 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5\": container with ID starting with 8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5 not found: ID does not exist" containerID="8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5" Sep 17 13:27:54.072339 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.072245 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5"} err="failed to get container status \"8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5\": rpc error: code = NotFound desc = could not find container \"8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5\": container with ID starting with 8bc99f1833ebc1323b6f463f47caa81386d9f915169d0783314c85a2ebf4dad5 not found: ID does not exist" Sep 17 13:27:54.072339 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.072260 2575 scope.go:117] "RemoveContainer" containerID="030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3" Sep 17 13:27:54.079163 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.079149 2575 scope.go:117] "RemoveContainer" containerID="030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3" Sep 17 13:27:54.079486 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:27:54.079463 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3\": container with ID starting with 030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3 not found: ID does not exist" containerID="030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3" Sep 17 13:27:54.079611 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.079490 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3"} err="failed to get container status \"030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3\": rpc error: code = NotFound desc = could not find container \"030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3\": container with ID starting with 030a876eb81f67463001df015b18a395c4bea67555cebc597fd2c33f6efc8ec3 not found: ID does not exist" Sep 17 13:27:54.081444 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.081402 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-62rx2"] Sep 17 13:27:54.084373 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.084354 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-62rx2"] Sep 17 13:27:54.095552 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.095533 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-ck26p"] Sep 17 13:27:54.098321 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:54.098305 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-ck26p"] Sep 17 13:27:55.611927 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:55.611892 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54c1c6b5-310f-4e92-9947-c9e4c9768991" path="/var/lib/kubelet/pods/54c1c6b5-310f-4e92-9947-c9e4c9768991/volumes" Sep 17 13:27:55.612457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:27:55.612433 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2324eda-2c55-402f-938d-9476b42a47d1" path="/var/lib/kubelet/pods/c2324eda-2c55-402f-938d-9476b42a47d1/volumes" Sep 17 13:28:03.759320 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:03.759262 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 17 13:28:13.759536 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:13.759453 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 17 13:28:13.759968 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:13.759563 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:28:23.186888 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.186853 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2d0e828-f962-47be-8330-8807472e4739" containerID="359500453df1f7206f9d4c9752a1bf60bbc060d744830e6251ec53b1c74deb49" exitCode=137 Sep 17 13:28:23.186888 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.186884 2575 generic.go:356] "Generic (PLEG): container finished" podID="b2d0e828-f962-47be-8330-8807472e4739" containerID="9cdddccf9e523f0ef70b37b51ad0588261698fe46ccc001af3c27916f5c3084d" exitCode=137 Sep 17 13:28:23.187246 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.186888 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerDied","Data":"359500453df1f7206f9d4c9752a1bf60bbc060d744830e6251ec53b1c74deb49"} Sep 17 13:28:23.187246 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.186948 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerDied","Data":"9cdddccf9e523f0ef70b37b51ad0588261698fe46ccc001af3c27916f5c3084d"} Sep 17 13:28:23.435870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.435837 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused" Sep 17 13:28:23.586751 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.586728 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:28:23.709612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.709538 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-offline-tls\") pod \"b2d0e828-f962-47be-8330-8807472e4739\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " Sep 17 13:28:23.709612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.709576 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-registry-tls\") pod \"b2d0e828-f962-47be-8330-8807472e4739\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " Sep 17 13:28:23.709612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.709614 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-online-tls\") pod \"b2d0e828-f962-47be-8330-8807472e4739\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " Sep 17 13:28:23.709878 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.709734 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2gxz\" (UniqueName: \"kubernetes.io/projected/b2d0e828-f962-47be-8330-8807472e4739-kube-api-access-j2gxz\") pod \"b2d0e828-f962-47be-8330-8807472e4739\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " Sep 17 13:28:23.709878 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.709790 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d0e828-f962-47be-8330-8807472e4739-feast-data\") pod \"b2d0e828-f962-47be-8330-8807472e4739\" (UID: \"b2d0e828-f962-47be-8330-8807472e4739\") " Sep 17 13:28:23.711786 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.711754 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-online-tls" (OuterVolumeSpecName: "online-tls") pod "b2d0e828-f962-47be-8330-8807472e4739" (UID: "b2d0e828-f962-47be-8330-8807472e4739"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:28:23.711895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.711787 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b2d0e828-f962-47be-8330-8807472e4739" (UID: "b2d0e828-f962-47be-8330-8807472e4739"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:28:23.711895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.711854 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d0e828-f962-47be-8330-8807472e4739-kube-api-access-j2gxz" (OuterVolumeSpecName: "kube-api-access-j2gxz") pod "b2d0e828-f962-47be-8330-8807472e4739" (UID: "b2d0e828-f962-47be-8330-8807472e4739"). InnerVolumeSpecName "kube-api-access-j2gxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:28:23.712033 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.712014 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b2d0e828-f962-47be-8330-8807472e4739" (UID: "b2d0e828-f962-47be-8330-8807472e4739"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:28:23.718593 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.718570 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d0e828-f962-47be-8330-8807472e4739-feast-data" (OuterVolumeSpecName: "feast-data") pod "b2d0e828-f962-47be-8330-8807472e4739" (UID: "b2d0e828-f962-47be-8330-8807472e4739"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:28:23.810421 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.810394 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:28:23.810421 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.810417 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:28:23.810421 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.810426 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b2d0e828-f962-47be-8330-8807472e4739-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:28:23.810568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.810435 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2gxz\" (UniqueName: \"kubernetes.io/projected/b2d0e828-f962-47be-8330-8807472e4739-kube-api-access-j2gxz\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:28:23.810568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:23.810444 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d0e828-f962-47be-8330-8807472e4739-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:28:24.193480 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.193450 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" Sep 17 13:28:24.193847 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.193445 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc" event={"ID":"b2d0e828-f962-47be-8330-8807472e4739","Type":"ContainerDied","Data":"a5b198323b822c0d451d268881582063e31aa5f01309af570a70d024de59e550"} Sep 17 13:28:24.193847 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.193590 2575 scope.go:117] "RemoveContainer" containerID="359500453df1f7206f9d4c9752a1bf60bbc060d744830e6251ec53b1c74deb49" Sep 17 13:28:24.202981 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.202793 2575 scope.go:117] "RemoveContainer" containerID="33d317a6ae9834e13322d1aa96f3969f54918680a3f5297b55cb7bdd834aa16a" Sep 17 13:28:24.210693 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.210674 2575 scope.go:117] "RemoveContainer" containerID="9cdddccf9e523f0ef70b37b51ad0588261698fe46ccc001af3c27916f5c3084d" Sep 17 13:28:24.217680 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.217664 2575 scope.go:117] "RemoveContainer" containerID="a7618b78863893da0b44159b2a3a67e50621ad6a15f14d7606f002beda1dfd98" Sep 17 13:28:24.218615 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.218593 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc"] Sep 17 13:28:24.222632 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.222612 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-544b86bc6-5h9pc"] Sep 17 13:28:24.225679 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:24.225665 2575 scope.go:117] "RemoveContainer" containerID="41bdbd05bae3a96912a52eed6dc17dedacc8b085d64428b45213b071c71f11d5" Sep 17 13:28:25.608474 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:28:25.608441 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d0e828-f962-47be-8330-8807472e4739" path="/var/lib/kubelet/pods/b2d0e828-f962-47be-8330-8807472e4739/volumes" Sep 17 13:29:51.538037 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:29:51.537957 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:29:51.538617 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:29:51.538528 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:30:56.508082 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508051 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-mhxb8"] Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508360 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="registry" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508372 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="registry" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508384 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2324eda-2c55-402f-938d-9476b42a47d1" containerName="redis" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508389 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2324eda-2c55-402f-938d-9476b42a47d1" containerName="redis" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508396 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="feast-init" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508402 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="feast-init" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508412 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="feast-apply" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508417 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="feast-apply" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508428 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c1c6b5-310f-4e92-9947-c9e4c9768991" containerName="postgres" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508433 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c1c6b5-310f-4e92-9947-c9e4c9768991" containerName="postgres" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508441 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508447 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508455 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="offline" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508459 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="offline" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508504 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="online" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508516 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2324eda-2c55-402f-938d-9476b42a47d1" containerName="redis" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508521 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c1c6b5-310f-4e92-9947-c9e4c9768991" containerName="postgres" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508526 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="offline" Sep 17 13:30:56.510436 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.508530 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d0e828-f962-47be-8330-8807472e4739" containerName="registry" Sep 17 13:30:56.511295 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.511265 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" Sep 17 13:30:56.513215 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.513190 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 17 13:30:56.513334 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.513321 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 17 13:30:56.513721 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.513701 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-k724l\"" Sep 17 13:30:56.526892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.526870 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-mhxb8"] Sep 17 13:30:56.605542 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.605510 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpsf5\" (UniqueName: \"kubernetes.io/projected/b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2-kube-api-access-mpsf5\") pod \"redis-69bbd798b6-mhxb8\" (UID: \"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2\") " pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" Sep 17 13:30:56.706288 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.706251 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpsf5\" (UniqueName: \"kubernetes.io/projected/b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2-kube-api-access-mpsf5\") pod \"redis-69bbd798b6-mhxb8\" (UID: \"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2\") " pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" Sep 17 13:30:56.728070 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.728042 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpsf5\" (UniqueName: \"kubernetes.io/projected/b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2-kube-api-access-mpsf5\") pod \"redis-69bbd798b6-mhxb8\" (UID: \"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2\") " pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" Sep 17 13:30:56.833174 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:56.833139 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" Sep 17 13:30:57.109903 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:57.109793 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-mhxb8"] Sep 17 13:30:57.112362 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:30:57.112326 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6c6b8e1_8540_45f9_b062_8f2ffdc0a4b2.slice/crio-28421c2600638fa60702b2d83f3adcd70017ccde8e4e7dd66f8eed722dea3894 WatchSource:0}: Error finding container 28421c2600638fa60702b2d83f3adcd70017ccde8e4e7dd66f8eed722dea3894: Status 404 returned error can't find the container with id 28421c2600638fa60702b2d83f3adcd70017ccde8e4e7dd66f8eed722dea3894 Sep 17 13:30:57.779468 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:57.779435 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" event={"ID":"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2","Type":"ContainerStarted","Data":"c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9"} Sep 17 13:30:57.779851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:57.779480 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" event={"ID":"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2","Type":"ContainerStarted","Data":"28421c2600638fa60702b2d83f3adcd70017ccde8e4e7dd66f8eed722dea3894"} Sep 17 13:30:57.798401 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:30:57.798351 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" podStartSLOduration=1.798332742 podStartE2EDuration="1.798332742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:30:57.797629788 +0000 UTC m=+966.702198489" watchObservedRunningTime="2026-09-17 13:30:57.798332742 +0000 UTC m=+966.702901419" Sep 17 13:31:01.995007 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:01.994976 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2sppd"] Sep 17 13:31:01.998363 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:01.998342 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:31:02.000013 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:01.999987 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 17 13:31:02.005294 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.005257 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2sppd"] Sep 17 13:31:02.051911 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.051872 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v658q\" (UniqueName: \"kubernetes.io/projected/7678dc4b-0a36-460b-8374-489cf3cd6b1d-kube-api-access-v658q\") pod \"postgres-864846d4f5-2sppd\" (UID: \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\") " pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:31:02.052077 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.051921 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7678dc4b-0a36-460b-8374-489cf3cd6b1d-postgresdata\") pod \"postgres-864846d4f5-2sppd\" (UID: \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\") " pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:31:02.153023 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.152985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v658q\" (UniqueName: \"kubernetes.io/projected/7678dc4b-0a36-460b-8374-489cf3cd6b1d-kube-api-access-v658q\") pod \"postgres-864846d4f5-2sppd\" (UID: \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\") " pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:31:02.153321 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.153063 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7678dc4b-0a36-460b-8374-489cf3cd6b1d-postgresdata\") pod \"postgres-864846d4f5-2sppd\" (UID: \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\") " pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:31:02.153444 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.153424 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7678dc4b-0a36-460b-8374-489cf3cd6b1d-postgresdata\") pod \"postgres-864846d4f5-2sppd\" (UID: \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\") " pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:31:02.163586 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.163561 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v658q\" (UniqueName: \"kubernetes.io/projected/7678dc4b-0a36-460b-8374-489cf3cd6b1d-kube-api-access-v658q\") pod \"postgres-864846d4f5-2sppd\" (UID: \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\") " pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:31:02.309185 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.309150 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:31:02.583072 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.583048 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2sppd"] Sep 17 13:31:02.585658 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:31:02.585634 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7678dc4b_0a36_460b_8374_489cf3cd6b1d.slice/crio-dcbb299c3719101c28ff74a7d997d01e8097a2580ca57acc8232737588e0d9a4 WatchSource:0}: Error finding container dcbb299c3719101c28ff74a7d997d01e8097a2580ca57acc8232737588e0d9a4: Status 404 returned error can't find the container with id dcbb299c3719101c28ff74a7d997d01e8097a2580ca57acc8232737588e0d9a4 Sep 17 13:31:02.801450 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.801415 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" event={"ID":"7678dc4b-0a36-460b-8374-489cf3cd6b1d","Type":"ContainerStarted","Data":"b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f"} Sep 17 13:31:02.801450 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.801458 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" event={"ID":"7678dc4b-0a36-460b-8374-489cf3cd6b1d","Type":"ContainerStarted","Data":"dcbb299c3719101c28ff74a7d997d01e8097a2580ca57acc8232737588e0d9a4"} Sep 17 13:31:02.819354 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:02.819306 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" podStartSLOduration=1.81928954 podStartE2EDuration="1.81928954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:31:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:31:02.818245246 +0000 UTC m=+971.722813923" watchObservedRunningTime="2026-09-17 13:31:02.81928954 +0000 UTC m=+971.723858211" Sep 17 13:31:07.711225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.711187 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc"] Sep 17 13:31:07.714727 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.714708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.716584 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.716551 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 17 13:31:07.716584 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.716578 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 17 13:31:07.716727 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.716600 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 17 13:31:07.716727 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.716565 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-qhzxc\"" Sep 17 13:31:07.725442 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.725421 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc"] Sep 17 13:31:07.796140 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.796109 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4af7cca9-bb42-4e69-b738-4dc7546787a7-feast-data\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.796316 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.796149 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-online-tls\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.796316 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.796210 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-registry-tls\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.796417 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.796325 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brmkg\" (UniqueName: \"kubernetes.io/projected/4af7cca9-bb42-4e69-b738-4dc7546787a7-kube-api-access-brmkg\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.897044 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.897002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brmkg\" (UniqueName: \"kubernetes.io/projected/4af7cca9-bb42-4e69-b738-4dc7546787a7-kube-api-access-brmkg\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.897233 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.897069 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4af7cca9-bb42-4e69-b738-4dc7546787a7-feast-data\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.897233 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.897091 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-online-tls\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.897233 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.897126 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-registry-tls\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.897523 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.897504 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4af7cca9-bb42-4e69-b738-4dc7546787a7-feast-data\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.899827 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.899801 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-registry-tls\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.899926 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.899838 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-online-tls\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:07.907091 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:07.907072 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brmkg\" (UniqueName: \"kubernetes.io/projected/4af7cca9-bb42-4e69-b738-4dc7546787a7-kube-api-access-brmkg\") pod \"feast-credit-scoring-5788fccc97-v8vqc\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:08.024066 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:08.024029 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:08.298964 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:08.298942 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc"] Sep 17 13:31:08.300906 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:31:08.300881 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af7cca9_bb42_4e69_b738_4dc7546787a7.slice/crio-43a17957a4bac871432970ac3bf3202a967bdee2cb38ea43835a3f4607c93156 WatchSource:0}: Error finding container 43a17957a4bac871432970ac3bf3202a967bdee2cb38ea43835a3f4607c93156: Status 404 returned error can't find the container with id 43a17957a4bac871432970ac3bf3202a967bdee2cb38ea43835a3f4607c93156 Sep 17 13:31:08.826919 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:08.826890 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerStarted","Data":"f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1"} Sep 17 13:31:08.827300 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:08.826933 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerStarted","Data":"43a17957a4bac871432970ac3bf3202a967bdee2cb38ea43835a3f4607c93156"} Sep 17 13:31:10.836435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:10.836401 2575 generic.go:356] "Generic (PLEG): container finished" podID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerID="f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1" exitCode=0 Sep 17 13:31:10.836892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:10.836481 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerDied","Data":"f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1"} Sep 17 13:31:11.843700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:11.843669 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerStarted","Data":"bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b"} Sep 17 13:31:17.869758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:17.869681 2575 generic.go:356] "Generic (PLEG): container finished" podID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerID="bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b" exitCode=0 Sep 17 13:31:17.870136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:17.869752 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerDied","Data":"bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b"} Sep 17 13:31:18.877711 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:18.877682 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerStarted","Data":"e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed"} Sep 17 13:31:18.878164 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:18.877743 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerStarted","Data":"8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5"} Sep 17 13:31:18.896844 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:18.896797 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podStartSLOduration=11.896778334 podStartE2EDuration="11.896778334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:31:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:31:18.894941296 +0000 UTC m=+987.799509990" watchObservedRunningTime="2026-09-17 13:31:18.896778334 +0000 UTC m=+987.801347012" Sep 17 13:31:20.025121 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:20.025077 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:20.025509 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:20.025154 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:20.026703 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:20.026668 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Sep 17 13:31:20.026703 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:20.026697 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 13:31:23.024906 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:23.024868 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 13:31:23.025309 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:23.024866 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Sep 17 13:31:26.025118 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:26.025069 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Sep 17 13:31:26.030658 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:26.030633 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:26.031004 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:26.030985 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:26.033935 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:26.033908 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:28.024284 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:28.024230 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:29.024984 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:29.024944 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Sep 17 13:31:32.025329 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:32.025300 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:32.026002 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:32.025985 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:31:33.647798 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.647764 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz"] Sep 17 13:31:33.652365 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.652344 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.654180 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.654160 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 17 13:31:33.654475 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.654453 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 17 13:31:33.654475 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.654472 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 17 13:31:33.654638 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.654480 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-q7h7m\"" Sep 17 13:31:33.669930 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.669911 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz"] Sep 17 13:31:33.821412 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.821370 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.821586 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.821471 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk2gf\" (UniqueName: \"kubernetes.io/projected/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-kube-api-access-zk2gf\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.821586 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.821507 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-online-tls\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.821693 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.821588 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-feast-data\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.922622 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.922552 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-feast-data\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.922622 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.922593 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.922809 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:31:33.922683 2575 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found Sep 17 13:31:33.922809 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.922718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk2gf\" (UniqueName: \"kubernetes.io/projected/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-kube-api-access-zk2gf\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.922809 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:31:33.922742 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls podName:74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f nodeName:}" failed. No retries permitted until 2026-09-17 13:31:34.42272771 +0000 UTC m=+1003.327296366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls") pod "feast-driver-ranking-7c45594bb6-m8ktz" (UID: "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f") : secret "feast-driver-ranking-registry-tls" not found Sep 17 13:31:33.922809 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.922762 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-online-tls\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.922983 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.922883 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-feast-data\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.924995 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.924979 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-online-tls\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:33.944085 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:33.944058 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk2gf\" (UniqueName: \"kubernetes.io/projected/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-kube-api-access-zk2gf\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:34.426114 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:34.426016 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:34.428429 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:34.428405 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls\") pod \"feast-driver-ranking-7c45594bb6-m8ktz\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:34.560889 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:34.560862 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:34.850608 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:34.850568 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz"] Sep 17 13:31:34.853520 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:31:34.853498 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74086e2f_bff3_4ed0_8418_8cdcfe4d9a5f.slice/crio-4d8afbb1b00721ea640e26ab0c97d34df0c3e668b572911cb2ef96afd4e5957d WatchSource:0}: Error finding container 4d8afbb1b00721ea640e26ab0c97d34df0c3e668b572911cb2ef96afd4e5957d: Status 404 returned error can't find the container with id 4d8afbb1b00721ea640e26ab0c97d34df0c3e668b572911cb2ef96afd4e5957d Sep 17 13:31:34.947697 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:34.947661 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerStarted","Data":"b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410"} Sep 17 13:31:34.947857 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:34.947715 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerStarted","Data":"4d8afbb1b00721ea640e26ab0c97d34df0c3e668b572911cb2ef96afd4e5957d"} Sep 17 13:31:39.968647 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:39.968615 2575 generic.go:356] "Generic (PLEG): container finished" podID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerID="b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410" exitCode=0 Sep 17 13:31:39.969131 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:39.968654 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerDied","Data":"b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410"} Sep 17 13:31:40.981953 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:40.981916 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerStarted","Data":"537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa"} Sep 17 13:31:47.010513 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:47.010480 2575 generic.go:356] "Generic (PLEG): container finished" podID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerID="537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa" exitCode=0 Sep 17 13:31:47.010879 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:47.010562 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerDied","Data":"537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa"} Sep 17 13:31:48.019116 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:48.019082 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerStarted","Data":"d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af"} Sep 17 13:31:48.019619 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:48.019140 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerStarted","Data":"8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a"} Sep 17 13:31:48.044934 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:48.044864 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podStartSLOduration=15.044846856 podStartE2EDuration="15.044846856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:31:48.043159274 +0000 UTC m=+1016.947727973" watchObservedRunningTime="2026-09-17 13:31:48.044846856 +0000 UTC m=+1016.949415534" Sep 17 13:31:49.561924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:49.561881 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:49.562342 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:49.561970 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:49.563481 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:49.563445 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.34:6571: connect: connection refused" Sep 17 13:31:49.563623 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:49.563487 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused" Sep 17 13:31:52.562336 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:52.562286 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.34:6571: connect: connection refused" Sep 17 13:31:52.562900 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:52.562296 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused" Sep 17 13:31:54.561793 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:54.561744 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:54.562176 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:54.561840 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:55.562035 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:55.561994 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.34:6571: connect: connection refused" Sep 17 13:31:55.567997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:55.567971 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:56.061762 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:56.061735 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:58.561895 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:58.561864 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:31:58.562751 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:31:58.562730 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:33:16.421160 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.421130 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx"] Sep 17 13:33:16.421644 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.421343 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" podUID="59d197d5-ca5e-4828-b158-cb660adc6b33" containerName="manager" containerID="cri-o://dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953" gracePeriod=10 Sep 17 13:33:16.680254 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.680190 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:33:16.684902 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.684879 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc"] Sep 17 13:33:16.685209 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.685080 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="registry" containerID="cri-o://8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5" gracePeriod=30 Sep 17 13:33:16.685209 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.685093 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" containerID="cri-o://e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed" gracePeriod=30 Sep 17 13:33:16.689161 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.689137 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz"] Sep 17 13:33:16.689429 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.689401 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="registry" containerID="cri-o://8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a" gracePeriod=30 Sep 17 13:33:16.689550 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.689423 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" containerID="cri-o://d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af" gracePeriod=30 Sep 17 13:33:16.799122 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.799094 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-599r7\" (UniqueName: \"kubernetes.io/projected/59d197d5-ca5e-4828-b158-cb660adc6b33-kube-api-access-599r7\") pod \"59d197d5-ca5e-4828-b158-cb660adc6b33\" (UID: \"59d197d5-ca5e-4828-b158-cb660adc6b33\") " Sep 17 13:33:16.801231 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.801210 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d197d5-ca5e-4828-b158-cb660adc6b33-kube-api-access-599r7" (OuterVolumeSpecName: "kube-api-access-599r7") pod "59d197d5-ca5e-4828-b158-cb660adc6b33" (UID: "59d197d5-ca5e-4828-b158-cb660adc6b33"). InnerVolumeSpecName "kube-api-access-599r7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:33:16.900689 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:16.900660 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-599r7\" (UniqueName: \"kubernetes.io/projected/59d197d5-ca5e-4828-b158-cb660adc6b33-kube-api-access-599r7\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:17.414912 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.414876 2575 generic.go:356] "Generic (PLEG): container finished" podID="59d197d5-ca5e-4828-b158-cb660adc6b33" containerID="dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953" exitCode=0 Sep 17 13:33:17.415086 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.414934 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" Sep 17 13:33:17.415086 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.414967 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" event={"ID":"59d197d5-ca5e-4828-b158-cb660adc6b33","Type":"ContainerDied","Data":"dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953"} Sep 17 13:33:17.415086 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.415024 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx" event={"ID":"59d197d5-ca5e-4828-b158-cb660adc6b33","Type":"ContainerDied","Data":"a5f87ac099e2f076783ab5270b918f8bf3b7ba06658b1390f381d61683262d21"} Sep 17 13:33:17.415086 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.415046 2575 scope.go:117] "RemoveContainer" containerID="dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953" Sep 17 13:33:17.424708 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.424377 2575 scope.go:117] "RemoveContainer" containerID="dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953" Sep 17 13:33:17.425045 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:17.424768 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953\": container with ID starting with dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953 not found: ID does not exist" containerID="dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953" Sep 17 13:33:17.425045 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.424809 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953"} err="failed to get container status \"dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953\": rpc error: code = NotFound desc = could not find container \"dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953\": container with ID starting with dcbda05f64dc0fd2986ab75896262dfcf97e925859a3dfae78a5abb9402eb953 not found: ID does not exist" Sep 17 13:33:17.437713 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.437688 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx"] Sep 17 13:33:17.445147 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.445124 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-68d9d5bf96-t5jqx"] Sep 17 13:33:17.608221 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:17.608190 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59d197d5-ca5e-4828-b158-cb660adc6b33" path="/var/lib/kubelet/pods/59d197d5-ca5e-4828-b158-cb660adc6b33/volumes" Sep 17 13:33:18.422916 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:18.422882 2575 generic.go:356] "Generic (PLEG): container finished" podID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerID="e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed" exitCode=0 Sep 17 13:33:18.423093 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:18.422949 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerDied","Data":"e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed"} Sep 17 13:33:18.425107 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:18.425087 2575 generic.go:356] "Generic (PLEG): container finished" podID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerID="d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af" exitCode=0 Sep 17 13:33:18.425434 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:18.425142 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerDied","Data":"d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af"} Sep 17 13:33:19.066612 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.066577 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2sppd"] Sep 17 13:33:19.066795 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.066759 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" podUID="7678dc4b-0a36-460b-8374-489cf3cd6b1d" containerName="postgres" containerID="cri-o://b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f" gracePeriod=30 Sep 17 13:33:19.072536 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.072507 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-mhxb8"] Sep 17 13:33:19.072702 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.072675 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" podUID="b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2" containerName="redis" containerID="cri-o://c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9" gracePeriod=30 Sep 17 13:33:19.326699 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.326674 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:33:19.329090 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.329074 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" Sep 17 13:33:19.421289 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.421242 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v658q\" (UniqueName: \"kubernetes.io/projected/7678dc4b-0a36-460b-8374-489cf3cd6b1d-kube-api-access-v658q\") pod \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\" (UID: \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\") " Sep 17 13:33:19.421435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.421304 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpsf5\" (UniqueName: \"kubernetes.io/projected/b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2-kube-api-access-mpsf5\") pod \"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2\" (UID: \"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2\") " Sep 17 13:33:19.421435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.421360 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7678dc4b-0a36-460b-8374-489cf3cd6b1d-postgresdata\") pod \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\" (UID: \"7678dc4b-0a36-460b-8374-489cf3cd6b1d\") " Sep 17 13:33:19.423466 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.423441 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2-kube-api-access-mpsf5" (OuterVolumeSpecName: "kube-api-access-mpsf5") pod "b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2" (UID: "b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2"). InnerVolumeSpecName "kube-api-access-mpsf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:33:19.423563 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.423477 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7678dc4b-0a36-460b-8374-489cf3cd6b1d-kube-api-access-v658q" (OuterVolumeSpecName: "kube-api-access-v658q") pod "7678dc4b-0a36-460b-8374-489cf3cd6b1d" (UID: "7678dc4b-0a36-460b-8374-489cf3cd6b1d"). InnerVolumeSpecName "kube-api-access-v658q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:33:19.430208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.430184 2575 generic.go:356] "Generic (PLEG): container finished" podID="b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2" containerID="c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9" exitCode=0 Sep 17 13:33:19.430590 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.430230 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" Sep 17 13:33:19.430590 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.430262 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" event={"ID":"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2","Type":"ContainerDied","Data":"c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9"} Sep 17 13:33:19.430590 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.430319 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-mhxb8" event={"ID":"b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2","Type":"ContainerDied","Data":"28421c2600638fa60702b2d83f3adcd70017ccde8e4e7dd66f8eed722dea3894"} Sep 17 13:33:19.430590 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.430343 2575 scope.go:117] "RemoveContainer" containerID="c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9" Sep 17 13:33:19.431686 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.431658 2575 generic.go:356] "Generic (PLEG): container finished" podID="7678dc4b-0a36-460b-8374-489cf3cd6b1d" containerID="b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f" exitCode=0 Sep 17 13:33:19.431811 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.431694 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" Sep 17 13:33:19.431811 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.431700 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" event={"ID":"7678dc4b-0a36-460b-8374-489cf3cd6b1d","Type":"ContainerDied","Data":"b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f"} Sep 17 13:33:19.431811 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.431730 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-2sppd" event={"ID":"7678dc4b-0a36-460b-8374-489cf3cd6b1d","Type":"ContainerDied","Data":"dcbb299c3719101c28ff74a7d997d01e8097a2580ca57acc8232737588e0d9a4"} Sep 17 13:33:19.439241 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.439218 2575 scope.go:117] "RemoveContainer" containerID="c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9" Sep 17 13:33:19.439496 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:19.439478 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9\": container with ID starting with c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9 not found: ID does not exist" containerID="c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9" Sep 17 13:33:19.439576 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.439508 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9"} err="failed to get container status \"c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9\": rpc error: code = NotFound desc = could not find container \"c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9\": container with ID starting with c4f1498b6d760d3dfa4ef44cc26c12035bbe9e845db0ed87ecbfca13bbc617b9 not found: ID does not exist" Sep 17 13:33:19.439576 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.439531 2575 scope.go:117] "RemoveContainer" containerID="b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f" Sep 17 13:33:19.446828 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.446808 2575 scope.go:117] "RemoveContainer" containerID="b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f" Sep 17 13:33:19.447093 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:19.447075 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f\": container with ID starting with b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f not found: ID does not exist" containerID="b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f" Sep 17 13:33:19.447152 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.447102 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f"} err="failed to get container status \"b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f\": rpc error: code = NotFound desc = could not find container \"b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f\": container with ID starting with b24697ae391f5180fef08601e65d1aad7e669f0852e8e28df5c777821a73817f not found: ID does not exist" Sep 17 13:33:19.450126 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.450104 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-mhxb8"] Sep 17 13:33:19.452205 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.452183 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7678dc4b-0a36-460b-8374-489cf3cd6b1d-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "7678dc4b-0a36-460b-8374-489cf3cd6b1d" (UID: "7678dc4b-0a36-460b-8374-489cf3cd6b1d"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:33:19.454058 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.454039 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-mhxb8"] Sep 17 13:33:19.522452 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.522424 2575 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7678dc4b-0a36-460b-8374-489cf3cd6b1d-postgresdata\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:19.522452 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.522449 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v658q\" (UniqueName: \"kubernetes.io/projected/7678dc4b-0a36-460b-8374-489cf3cd6b1d-kube-api-access-v658q\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:19.522452 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.522459 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpsf5\" (UniqueName: \"kubernetes.io/projected/b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2-kube-api-access-mpsf5\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:19.607217 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.607146 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2" path="/var/lib/kubelet/pods/b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2/volumes" Sep 17 13:33:19.747224 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.747200 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2sppd"] Sep 17 13:33:19.753893 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:19.753869 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2sppd"] Sep 17 13:33:21.607079 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:21.607038 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7678dc4b-0a36-460b-8374-489cf3cd6b1d" path="/var/lib/kubelet/pods/7678dc4b-0a36-460b-8374-489cf3cd6b1d/volumes" Sep 17 13:33:26.031747 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:26.031708 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 13:33:26.059340 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:26.059306 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused" Sep 17 13:33:27.828453 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828419 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf"] Sep 17 13:33:27.828824 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828761 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59d197d5-ca5e-4828-b158-cb660adc6b33" containerName="manager" Sep 17 13:33:27.828824 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828773 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="59d197d5-ca5e-4828-b158-cb660adc6b33" containerName="manager" Sep 17 13:33:27.828824 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828785 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2" containerName="redis" Sep 17 13:33:27.828824 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828790 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2" containerName="redis" Sep 17 13:33:27.828824 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828805 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7678dc4b-0a36-460b-8374-489cf3cd6b1d" containerName="postgres" Sep 17 13:33:27.828824 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828814 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7678dc4b-0a36-460b-8374-489cf3cd6b1d" containerName="postgres" Sep 17 13:33:27.829051 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828882 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6c6b8e1-8540-45f9-b062-8f2ffdc0a4b2" containerName="redis" Sep 17 13:33:27.829051 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828892 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7678dc4b-0a36-460b-8374-489cf3cd6b1d" containerName="postgres" Sep 17 13:33:27.829051 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.828906 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="59d197d5-ca5e-4828-b158-cb660adc6b33" containerName="manager" Sep 17 13:33:27.835874 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.835852 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:33:27.837889 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.837871 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 13:33:27.837995 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.837897 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 13:33:27.838450 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.838436 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-dqtft\"" Sep 17 13:33:27.847122 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.847103 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf"] Sep 17 13:33:27.886680 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.886641 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s6jr\" (UniqueName: \"kubernetes.io/projected/176770f7-eb60-42d1-9cc5-b96f66366127-kube-api-access-7s6jr\") pod \"feast-operator-controller-manager-689b877b4b-k6tdf\" (UID: \"176770f7-eb60-42d1-9cc5-b96f66366127\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:33:27.987841 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.987804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s6jr\" (UniqueName: \"kubernetes.io/projected/176770f7-eb60-42d1-9cc5-b96f66366127-kube-api-access-7s6jr\") pod \"feast-operator-controller-manager-689b877b4b-k6tdf\" (UID: \"176770f7-eb60-42d1-9cc5-b96f66366127\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:33:27.998997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:27.998972 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s6jr\" (UniqueName: \"kubernetes.io/projected/176770f7-eb60-42d1-9cc5-b96f66366127-kube-api-access-7s6jr\") pod \"feast-operator-controller-manager-689b877b4b-k6tdf\" (UID: \"176770f7-eb60-42d1-9cc5-b96f66366127\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:33:28.145837 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:28.145757 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:33:28.416675 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:28.416608 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf"] Sep 17 13:33:28.419931 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:33:28.419900 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod176770f7_eb60_42d1_9cc5_b96f66366127.slice/crio-c852a038fd284ebbd01c8389945ce701f138218b43d1335226bc358c55bd5f92 WatchSource:0}: Error finding container c852a038fd284ebbd01c8389945ce701f138218b43d1335226bc358c55bd5f92: Status 404 returned error can't find the container with id c852a038fd284ebbd01c8389945ce701f138218b43d1335226bc358c55bd5f92 Sep 17 13:33:28.421821 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:28.421805 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 13:33:28.469360 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:28.469334 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" event={"ID":"176770f7-eb60-42d1-9cc5-b96f66366127","Type":"ContainerStarted","Data":"c852a038fd284ebbd01c8389945ce701f138218b43d1335226bc358c55bd5f92"} Sep 17 13:33:31.484966 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:31.484934 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" event={"ID":"176770f7-eb60-42d1-9cc5-b96f66366127","Type":"ContainerStarted","Data":"a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279"} Sep 17 13:33:31.485371 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:31.485059 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:33:31.503637 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:31.503586 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" podStartSLOduration=2.190918939 podStartE2EDuration="4.503568229s" totalImagesPullingTime="2.31264929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:33:31.503023963 +0000 UTC m=+1120.407592665" watchObservedRunningTime="2026-09-17 13:33:31.503568229 +0000 UTC m=+1120.408136907" Sep 17 13:33:36.031950 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:36.031907 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 13:33:36.059036 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:36.059004 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused" Sep 17 13:33:42.490722 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:42.490693 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:33:46.031508 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:46.031463 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Sep 17 13:33:46.031871 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:46.031587 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:33:46.058748 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:46.058718 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused" Sep 17 13:33:46.058880 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:46.058800 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:33:47.335770 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.335748 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:33:47.358004 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.357974 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:33:47.442731 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.442701 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brmkg\" (UniqueName: \"kubernetes.io/projected/4af7cca9-bb42-4e69-b738-4dc7546787a7-kube-api-access-brmkg\") pod \"4af7cca9-bb42-4e69-b738-4dc7546787a7\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " Sep 17 13:33:47.442897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.442743 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4af7cca9-bb42-4e69-b738-4dc7546787a7-feast-data\") pod \"4af7cca9-bb42-4e69-b738-4dc7546787a7\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " Sep 17 13:33:47.442897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.442764 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-feast-data\") pod \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " Sep 17 13:33:47.442897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.442789 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk2gf\" (UniqueName: \"kubernetes.io/projected/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-kube-api-access-zk2gf\") pod \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " Sep 17 13:33:47.442897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.442845 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-online-tls\") pod \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " Sep 17 13:33:47.442897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.442894 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-registry-tls\") pod \"4af7cca9-bb42-4e69-b738-4dc7546787a7\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " Sep 17 13:33:47.443236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.442920 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-online-tls\") pod \"4af7cca9-bb42-4e69-b738-4dc7546787a7\" (UID: \"4af7cca9-bb42-4e69-b738-4dc7546787a7\") " Sep 17 13:33:47.443236 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.442954 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls\") pod \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\" (UID: \"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f\") " Sep 17 13:33:47.443377 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.443351 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-feast-data" (OuterVolumeSpecName: "feast-data") pod "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" (UID: "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:33:47.445134 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.445100 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-kube-api-access-zk2gf" (OuterVolumeSpecName: "kube-api-access-zk2gf") pod "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" (UID: "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f"). InnerVolumeSpecName "kube-api-access-zk2gf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:33:47.445630 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.445600 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" (UID: "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:33:47.445716 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.445641 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-online-tls" (OuterVolumeSpecName: "online-tls") pod "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" (UID: "74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:33:47.445716 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.445684 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af7cca9-bb42-4e69-b738-4dc7546787a7-kube-api-access-brmkg" (OuterVolumeSpecName: "kube-api-access-brmkg") pod "4af7cca9-bb42-4e69-b738-4dc7546787a7" (UID: "4af7cca9-bb42-4e69-b738-4dc7546787a7"). InnerVolumeSpecName "kube-api-access-brmkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:33:47.445825 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.445786 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-online-tls" (OuterVolumeSpecName: "online-tls") pod "4af7cca9-bb42-4e69-b738-4dc7546787a7" (UID: "4af7cca9-bb42-4e69-b738-4dc7546787a7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:33:47.445864 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.445850 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4af7cca9-bb42-4e69-b738-4dc7546787a7" (UID: "4af7cca9-bb42-4e69-b738-4dc7546787a7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:33:47.452140 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.452096 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4af7cca9-bb42-4e69-b738-4dc7546787a7-feast-data" (OuterVolumeSpecName: "feast-data") pod "4af7cca9-bb42-4e69-b738-4dc7546787a7" (UID: "4af7cca9-bb42-4e69-b738-4dc7546787a7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:33:47.544435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.544406 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:47.544435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.544429 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:47.544435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.544437 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4af7cca9-bb42-4e69-b738-4dc7546787a7-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:47.544435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.544444 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:47.544662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.544452 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brmkg\" (UniqueName: \"kubernetes.io/projected/4af7cca9-bb42-4e69-b738-4dc7546787a7-kube-api-access-brmkg\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:47.544662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.544460 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4af7cca9-bb42-4e69-b738-4dc7546787a7-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:47.544662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.544468 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:47.544662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.544475 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk2gf\" (UniqueName: \"kubernetes.io/projected/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f-kube-api-access-zk2gf\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:33:47.551244 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.551218 2575 generic.go:356] "Generic (PLEG): container finished" podID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerID="8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5" exitCode=137 Sep 17 13:33:47.551385 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.551297 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" Sep 17 13:33:47.551385 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.551309 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerDied","Data":"8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5"} Sep 17 13:33:47.551385 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.551368 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc" event={"ID":"4af7cca9-bb42-4e69-b738-4dc7546787a7","Type":"ContainerDied","Data":"43a17957a4bac871432970ac3bf3202a967bdee2cb38ea43835a3f4607c93156"} Sep 17 13:33:47.551541 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.551395 2575 scope.go:117] "RemoveContainer" containerID="e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed" Sep 17 13:33:47.553796 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.553776 2575 generic.go:356] "Generic (PLEG): container finished" podID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerID="8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a" exitCode=137 Sep 17 13:33:47.553919 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.553830 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" Sep 17 13:33:47.553919 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.553832 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerDied","Data":"8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a"} Sep 17 13:33:47.553919 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.553883 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz" event={"ID":"74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f","Type":"ContainerDied","Data":"4d8afbb1b00721ea640e26ab0c97d34df0c3e668b572911cb2ef96afd4e5957d"} Sep 17 13:33:47.560565 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.560549 2575 scope.go:117] "RemoveContainer" containerID="8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5" Sep 17 13:33:47.568388 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.568372 2575 scope.go:117] "RemoveContainer" containerID="bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b" Sep 17 13:33:47.573820 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.573796 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc"] Sep 17 13:33:47.575812 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.575798 2575 scope.go:117] "RemoveContainer" containerID="f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1" Sep 17 13:33:47.580159 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.580143 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-5788fccc97-v8vqc"] Sep 17 13:33:47.583312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.583286 2575 scope.go:117] "RemoveContainer" containerID="e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed" Sep 17 13:33:47.583583 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:47.583565 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed\": container with ID starting with e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed not found: ID does not exist" containerID="e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed" Sep 17 13:33:47.583639 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.583589 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed"} err="failed to get container status \"e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed\": rpc error: code = NotFound desc = could not find container \"e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed\": container with ID starting with e255a35ecce5270be9dae7dd854744caa3481e9caf25116bcb26c298f9ba78ed not found: ID does not exist" Sep 17 13:33:47.583639 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.583606 2575 scope.go:117] "RemoveContainer" containerID="8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5" Sep 17 13:33:47.583847 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:47.583831 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5\": container with ID starting with 8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5 not found: ID does not exist" containerID="8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5" Sep 17 13:33:47.583904 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.583857 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5"} err="failed to get container status \"8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5\": rpc error: code = NotFound desc = could not find container \"8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5\": container with ID starting with 8a97819450b06037ac68920b6008709f03e34623a38d1a02ac4edc449f31c6d5 not found: ID does not exist" Sep 17 13:33:47.583904 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.583879 2575 scope.go:117] "RemoveContainer" containerID="bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b" Sep 17 13:33:47.584098 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:47.584082 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b\": container with ID starting with bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b not found: ID does not exist" containerID="bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b" Sep 17 13:33:47.584133 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.584103 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b"} err="failed to get container status \"bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b\": rpc error: code = NotFound desc = could not find container \"bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b\": container with ID starting with bc9c3b3df80114e434dec036faacfcd97a0cb09cf13e48a0aeccd941be996b3b not found: ID does not exist" Sep 17 13:33:47.584133 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.584115 2575 scope.go:117] "RemoveContainer" containerID="f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1" Sep 17 13:33:47.584338 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:47.584320 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1\": container with ID starting with f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1 not found: ID does not exist" containerID="f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1" Sep 17 13:33:47.584426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.584339 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1"} err="failed to get container status \"f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1\": rpc error: code = NotFound desc = could not find container \"f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1\": container with ID starting with f80c66e4ecdb3d1955dbf90b21d7297bf062ca698b388887b1cdf45cc82d15d1 not found: ID does not exist" Sep 17 13:33:47.584426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.584350 2575 scope.go:117] "RemoveContainer" containerID="d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af" Sep 17 13:33:47.593031 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.593011 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz"] Sep 17 13:33:47.594238 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.594223 2575 scope.go:117] "RemoveContainer" containerID="8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a" Sep 17 13:33:47.596237 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.596221 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7c45594bb6-m8ktz"] Sep 17 13:33:47.602095 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.602080 2575 scope.go:117] "RemoveContainer" containerID="537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa" Sep 17 13:33:47.607234 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.607216 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" path="/var/lib/kubelet/pods/4af7cca9-bb42-4e69-b738-4dc7546787a7/volumes" Sep 17 13:33:47.607773 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.607759 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" path="/var/lib/kubelet/pods/74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f/volumes" Sep 17 13:33:47.609200 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.609185 2575 scope.go:117] "RemoveContainer" containerID="b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410" Sep 17 13:33:47.615861 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.615847 2575 scope.go:117] "RemoveContainer" containerID="d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af" Sep 17 13:33:47.616127 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:47.616110 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af\": container with ID starting with d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af not found: ID does not exist" containerID="d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af" Sep 17 13:33:47.616180 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.616134 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af"} err="failed to get container status \"d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af\": rpc error: code = NotFound desc = could not find container \"d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af\": container with ID starting with d702f59550fb51ec80390a165c6f07f5d4f337cf081a6f3e34dfa71367fe89af not found: ID does not exist" Sep 17 13:33:47.616180 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.616151 2575 scope.go:117] "RemoveContainer" containerID="8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a" Sep 17 13:33:47.616412 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:47.616397 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a\": container with ID starting with 8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a not found: ID does not exist" containerID="8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a" Sep 17 13:33:47.616462 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.616416 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a"} err="failed to get container status \"8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a\": rpc error: code = NotFound desc = could not find container \"8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a\": container with ID starting with 8e403c81bc2a3fe5f80b5a5ef1535764953558285110521496ac3fc8dcea118a not found: ID does not exist" Sep 17 13:33:47.616462 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.616428 2575 scope.go:117] "RemoveContainer" containerID="537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa" Sep 17 13:33:47.616660 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:47.616645 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa\": container with ID starting with 537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa not found: ID does not exist" containerID="537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa" Sep 17 13:33:47.616704 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.616663 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa"} err="failed to get container status \"537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa\": rpc error: code = NotFound desc = could not find container \"537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa\": container with ID starting with 537eeef87ad0659ae9f3e7ac20dae62a15e4a0b9e5dfbae5643340458d64acfa not found: ID does not exist" Sep 17 13:33:47.616704 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.616674 2575 scope.go:117] "RemoveContainer" containerID="b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410" Sep 17 13:33:47.616894 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:47.616880 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410\": container with ID starting with b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410 not found: ID does not exist" containerID="b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410" Sep 17 13:33:47.616934 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:47.616899 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410"} err="failed to get container status \"b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410\": rpc error: code = NotFound desc = could not find container \"b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410\": container with ID starting with b660091fe60071c49dc3a36ff5b4525abe4aed374adb9978cef85d5efca74410 not found: ID does not exist" Sep 17 13:33:55.946918 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.946890 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r"] Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947179 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947190 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947197 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="feast-apply" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947202 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="feast-apply" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947210 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="registry" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947215 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="registry" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947229 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="feast-init" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947234 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="feast-init" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947243 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="feast-apply" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947248 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="feast-apply" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947254 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" Sep 17 13:33:55.947258 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947259 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" Sep 17 13:33:55.947624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947266 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="registry" Sep 17 13:33:55.947624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947286 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="registry" Sep 17 13:33:55.947624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947291 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="feast-init" Sep 17 13:33:55.947624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947297 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="feast-init" Sep 17 13:33:55.947624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947367 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="online" Sep 17 13:33:55.947624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947372 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="registry" Sep 17 13:33:55.947624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947380 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="74086e2f-bff3-4ed0-8418-8cdcfe4d9a5f" containerName="online" Sep 17 13:33:55.947624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.947385 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4af7cca9-bb42-4e69-b738-4dc7546787a7" containerName="registry" Sep 17 13:33:55.953681 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.953652 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:55.955995 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.955971 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 13:33:55.956149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.955981 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 13:33:55.956149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.956107 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 13:33:55.956461 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.956436 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 13:33:55.956633 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.956607 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 13:33:55.956743 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.956553 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 13:33:55.956743 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.956555 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hcmpg\"" Sep 17 13:33:55.963609 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:55.963589 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r"] Sep 17 13:33:56.009972 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.009948 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.010104 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.009986 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df383424-5b3c-4278-a7e0-7acc933c82d4-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.010104 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.010003 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmlwq\" (UniqueName: \"kubernetes.io/projected/df383424-5b3c-4278-a7e0-7acc933c82d4-kube-api-access-cmlwq\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.010104 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.010070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.010104 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.010096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.010303 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.010127 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.111293 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.111244 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.111435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.111300 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df383424-5b3c-4278-a7e0-7acc933c82d4-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.111435 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.111412 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmlwq\" (UniqueName: \"kubernetes.io/projected/df383424-5b3c-4278-a7e0-7acc933c82d4-kube-api-access-cmlwq\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.111530 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.111469 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.111530 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.111498 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.111530 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.111525 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.111683 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:56.111630 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 17 13:33:56.111683 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.111669 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df383424-5b3c-4278-a7e0-7acc933c82d4-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.111786 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:33:56.111716 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls podName:df383424-5b3c-4278-a7e0-7acc933c82d4 nodeName:}" failed. No retries permitted until 2026-09-17 13:33:56.611696117 +0000 UTC m=+1145.516264773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-dh42r" (UID: "df383424-5b3c-4278-a7e0-7acc933c82d4") : secret "feast-simple-feast-setup-ui-tls" not found Sep 17 13:33:56.113839 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.113812 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.113967 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.113842 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.114110 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.114096 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.119384 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.119362 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmlwq\" (UniqueName: \"kubernetes.io/projected/df383424-5b3c-4278-a7e0-7acc933c82d4-kube-api-access-cmlwq\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.615564 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.615529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.617977 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.617954 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-dh42r\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:56.880975 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:56.880895 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:33:57.156708 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:57.156686 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r"] Sep 17 13:33:57.158952 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:33:57.158927 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf383424_5b3c_4278_a7e0_7acc933c82d4.slice/crio-6e8a93d268b9b4c33807d00413d0e80785d295abcf1036366339d7c8353b99c5 WatchSource:0}: Error finding container 6e8a93d268b9b4c33807d00413d0e80785d295abcf1036366339d7c8353b99c5: Status 404 returned error can't find the container with id 6e8a93d268b9b4c33807d00413d0e80785d295abcf1036366339d7c8353b99c5 Sep 17 13:33:57.593706 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:33:57.593676 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerStarted","Data":"6e8a93d268b9b4c33807d00413d0e80785d295abcf1036366339d7c8353b99c5"} Sep 17 13:34:10.657081 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:10.656997 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerStarted","Data":"86e40b1eddd247d6252f882d2fb8306aca8a375350a256d5bd9cb09ce7f85759"} Sep 17 13:34:15.675678 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:15.675646 2575 generic.go:356] "Generic (PLEG): container finished" podID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerID="86e40b1eddd247d6252f882d2fb8306aca8a375350a256d5bd9cb09ce7f85759" exitCode=0 Sep 17 13:34:15.676061 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:15.675713 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerDied","Data":"86e40b1eddd247d6252f882d2fb8306aca8a375350a256d5bd9cb09ce7f85759"} Sep 17 13:34:16.682634 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:16.682599 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerStarted","Data":"ca266f741fd497b774006432a4dfeeca14800d2dc459591229ad045c0a250df1"} Sep 17 13:34:21.703994 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:21.703963 2575 generic.go:356] "Generic (PLEG): container finished" podID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerID="ca266f741fd497b774006432a4dfeeca14800d2dc459591229ad045c0a250df1" exitCode=0 Sep 17 13:34:21.704348 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:21.704008 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerDied","Data":"ca266f741fd497b774006432a4dfeeca14800d2dc459591229ad045c0a250df1"} Sep 17 13:34:22.720868 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:22.720827 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerStarted","Data":"b2513dea7830b2ab1b9c39c18781c47a876f3fde93a3e023ecb27db803d5c876"} Sep 17 13:34:22.721347 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:22.720891 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerStarted","Data":"6f61e55b85db790bfee7220ce6eef389628390c7e23659d086f11bdd1c0730d5"} Sep 17 13:34:22.721347 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:22.720913 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerStarted","Data":"933b14d95d14e52ceb48cd2e55ea7070ef35c7c9ec6af1a827260dd9fec08b8c"} Sep 17 13:34:22.721347 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:22.720933 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerStarted","Data":"50c7b6a2d8544de2741edce87b3e16aac33f7afccff8eece22cc173ec3b756c2"} Sep 17 13:34:22.745456 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:22.744891 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podStartSLOduration=14.593261878 podStartE2EDuration="27.744865934s" totalImagesPullingTime="13.151604056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:34:22.74301774 +0000 UTC m=+1171.647586431" watchObservedRunningTime="2026-09-17 13:34:22.744865934 +0000 UTC m=+1171.649434611" Sep 17 13:34:23.881640 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:23.881607 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:23.882189 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:23.882154 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:23.882189 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:23.882191 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:23.882412 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:23.882211 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:23.883779 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:23.883741 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused" Sep 17 13:34:23.883779 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:23.883763 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 13:34:23.883943 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:23.883798 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 13:34:23.883943 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:23.883841 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused" Sep 17 13:34:26.881829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:26.881787 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:26.882438 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:26.882402 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:26.882621 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:26.882069 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused" Sep 17 13:34:26.882716 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:26.882067 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused" Sep 17 13:34:26.882716 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:26.882067 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 13:34:26.882911 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:26.882890 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:26.882989 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:26.882932 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:26.883095 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:26.882415 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 13:34:29.882440 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:29.882408 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:29.882880 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:29.882600 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:29.882880 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:29.882738 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:29.883360 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:29.883339 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:29.883529 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:29.883509 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:29.883674 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:29.883659 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:29.887952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:29.887935 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:30.769061 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:30.769026 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:34:51.565500 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:51.565476 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:34:51.566424 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:34:51.566401 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:35:33.019204 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.019175 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r"] Sep 17 13:35:33.019803 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.019423 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="registry" containerID="cri-o://50c7b6a2d8544de2741edce87b3e16aac33f7afccff8eece22cc173ec3b756c2" gracePeriod=30 Sep 17 13:35:33.019803 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.019469 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" containerID="cri-o://b2513dea7830b2ab1b9c39c18781c47a876f3fde93a3e023ecb27db803d5c876" gracePeriod=30 Sep 17 13:35:33.019803 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.019490 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="offline" containerID="cri-o://6f61e55b85db790bfee7220ce6eef389628390c7e23659d086f11bdd1c0730d5" gracePeriod=30 Sep 17 13:35:33.019803 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.019502 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" containerID="cri-o://933b14d95d14e52ceb48cd2e55ea7070ef35c7c9ec6af1a827260dd9fec08b8c" gracePeriod=30 Sep 17 13:35:33.521119 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.521077 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt"] Sep 17 13:35:33.524816 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.524790 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.526591 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.526569 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-ckwn6\"" Sep 17 13:35:33.541765 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.541737 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt"] Sep 17 13:35:33.613731 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.613692 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.613905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.613764 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.613905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.613789 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.613905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.613813 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.613905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.613871 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be2d9892-1b19-453c-b273-ac50dae08515-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.613905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.613903 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlzgb\" (UniqueName: \"kubernetes.io/projected/be2d9892-1b19-453c-b273-ac50dae08515-kube-api-access-qlzgb\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.714662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.714630 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.714662 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.714662 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.714916 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.714683 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.714916 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.714699 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be2d9892-1b19-453c-b273-ac50dae08515-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.714916 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.714717 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzgb\" (UniqueName: \"kubernetes.io/projected/be2d9892-1b19-453c-b273-ac50dae08515-kube-api-access-qlzgb\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.714916 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.714797 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.715213 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.715170 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be2d9892-1b19-453c-b273-ac50dae08515-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.717411 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.717388 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.717509 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.717456 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.717753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.717734 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.717856 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.717844 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.726723 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.726702 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlzgb\" (UniqueName: \"kubernetes.io/projected/be2d9892-1b19-453c-b273-ac50dae08515-kube-api-access-qlzgb\") pod \"feast-simple-feast-setup-57576b8fcf-nbmtt\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:33.834490 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:33.834462 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:34.018156 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:34.018124 2575 generic.go:356] "Generic (PLEG): container finished" podID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerID="b2513dea7830b2ab1b9c39c18781c47a876f3fde93a3e023ecb27db803d5c876" exitCode=0 Sep 17 13:35:34.018156 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:34.018150 2575 generic.go:356] "Generic (PLEG): container finished" podID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerID="933b14d95d14e52ceb48cd2e55ea7070ef35c7c9ec6af1a827260dd9fec08b8c" exitCode=0 Sep 17 13:35:34.018391 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:34.018185 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerDied","Data":"b2513dea7830b2ab1b9c39c18781c47a876f3fde93a3e023ecb27db803d5c876"} Sep 17 13:35:34.018391 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:34.018224 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerDied","Data":"933b14d95d14e52ceb48cd2e55ea7070ef35c7c9ec6af1a827260dd9fec08b8c"} Sep 17 13:35:34.107174 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:34.107119 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt"] Sep 17 13:35:34.109316 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:35:34.109282 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe2d9892_1b19_453c_b273_ac50dae08515.slice/crio-e4e8ffa1f0d709b2fc6d113d26e04a865452ace4b167f4486e2b021a2ace8d11 WatchSource:0}: Error finding container e4e8ffa1f0d709b2fc6d113d26e04a865452ace4b167f4486e2b021a2ace8d11: Status 404 returned error can't find the container with id e4e8ffa1f0d709b2fc6d113d26e04a865452ace4b167f4486e2b021a2ace8d11 Sep 17 13:35:35.023387 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:35.023344 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerStarted","Data":"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82"} Sep 17 13:35:35.023583 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:35.023401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerStarted","Data":"e4e8ffa1f0d709b2fc6d113d26e04a865452ace4b167f4486e2b021a2ace8d11"} Sep 17 13:35:39.041176 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:39.041147 2575 generic.go:356] "Generic (PLEG): container finished" podID="be2d9892-1b19-453c-b273-ac50dae08515" containerID="d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82" exitCode=0 Sep 17 13:35:39.041550 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:39.041185 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerDied","Data":"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82"} Sep 17 13:35:39.883431 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:39.883383 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 13:35:40.047699 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:40.047668 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerStarted","Data":"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117"} Sep 17 13:35:40.767105 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:40.767056 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 13:35:44.599071 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:35:44.599044 2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe2d9892_1b19_453c_b273_ac50dae08515.slice/crio-25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117.scope\": RecentStats: unable to find data in memory cache]" Sep 17 13:35:45.070930 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:45.070897 2575 generic.go:356] "Generic (PLEG): container finished" podID="be2d9892-1b19-453c-b273-ac50dae08515" containerID="25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117" exitCode=0 Sep 17 13:35:45.071110 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:45.070970 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerDied","Data":"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117"} Sep 17 13:35:46.082186 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:46.082133 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerStarted","Data":"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea"} Sep 17 13:35:46.082681 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:46.082208 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerStarted","Data":"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27"} Sep 17 13:35:46.082681 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:46.082228 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerStarted","Data":"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e"} Sep 17 13:35:46.082681 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:46.082257 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerStarted","Data":"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1"} Sep 17 13:35:46.108336 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:46.108260 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podStartSLOduration=13.108222524 podStartE2EDuration="13.108222524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:35:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:35:46.105996069 +0000 UTC m=+1255.010564813" watchObservedRunningTime="2026-09-17 13:35:46.108222524 +0000 UTC m=+1255.012791202" Sep 17 13:35:48.835087 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:48.834992 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:48.835087 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:48.835054 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:48.835087 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:48.835073 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:48.835087 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:48.835093 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:48.837240 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:48.837134 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused" Sep 17 13:35:48.837240 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:48.837167 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 13:35:48.837240 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:48.837143 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.37:6571: connect: connection refused" Sep 17 13:35:48.837240 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:48.837205 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 13:35:49.884116 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:49.884057 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 13:35:50.766591 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:50.766539 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 13:35:51.835264 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:51.835215 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused" Sep 17 13:35:51.835733 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:51.835215 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.37:6571: connect: connection refused" Sep 17 13:35:51.835733 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:51.835215 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 13:35:51.835733 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:51.835448 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 13:35:53.835085 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:53.835052 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:53.835594 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:53.835116 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:53.835594 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:53.835138 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:53.835594 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:53.835153 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:54.836452 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:54.836418 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:54.836914 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:54.836531 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:54.836914 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:54.836743 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:54.837317 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:54.837297 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:54.837457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:54.837367 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:54.837513 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:54.837505 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:54.841776 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:54.841758 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:55.137931 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:55.137858 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:35:59.884011 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:59.883967 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused" Sep 17 13:35:59.886431 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:35:59.884110 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:36:00.766601 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:00.766560 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused" Sep 17 13:36:00.766771 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:00.766670 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:36:03.167907 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.167880 2575 generic.go:356] "Generic (PLEG): container finished" podID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerID="6f61e55b85db790bfee7220ce6eef389628390c7e23659d086f11bdd1c0730d5" exitCode=137 Sep 17 13:36:03.167907 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.167903 2575 generic.go:356] "Generic (PLEG): container finished" podID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerID="50c7b6a2d8544de2741edce87b3e16aac33f7afccff8eece22cc173ec3b756c2" exitCode=137 Sep 17 13:36:03.168291 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.167952 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerDied","Data":"6f61e55b85db790bfee7220ce6eef389628390c7e23659d086f11bdd1c0730d5"} Sep 17 13:36:03.168291 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.168001 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerDied","Data":"50c7b6a2d8544de2741edce87b3e16aac33f7afccff8eece22cc173ec3b756c2"} Sep 17 13:36:03.659321 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.659298 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:36:03.780580 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.780546 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmlwq\" (UniqueName: \"kubernetes.io/projected/df383424-5b3c-4278-a7e0-7acc933c82d4-kube-api-access-cmlwq\") pod \"df383424-5b3c-4278-a7e0-7acc933c82d4\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " Sep 17 13:36:03.780758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.780606 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-offline-tls\") pod \"df383424-5b3c-4278-a7e0-7acc933c82d4\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " Sep 17 13:36:03.780758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.780629 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-registry-tls\") pod \"df383424-5b3c-4278-a7e0-7acc933c82d4\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " Sep 17 13:36:03.780758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.780650 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls\") pod \"df383424-5b3c-4278-a7e0-7acc933c82d4\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " Sep 17 13:36:03.780758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.780678 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df383424-5b3c-4278-a7e0-7acc933c82d4-feast-data\") pod \"df383424-5b3c-4278-a7e0-7acc933c82d4\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " Sep 17 13:36:03.780758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.780721 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-online-tls\") pod \"df383424-5b3c-4278-a7e0-7acc933c82d4\" (UID: \"df383424-5b3c-4278-a7e0-7acc933c82d4\") " Sep 17 13:36:03.781495 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.781454 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df383424-5b3c-4278-a7e0-7acc933c82d4-feast-data" (OuterVolumeSpecName: "feast-data") pod "df383424-5b3c-4278-a7e0-7acc933c82d4" (UID: "df383424-5b3c-4278-a7e0-7acc933c82d4"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:36:03.783122 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.783068 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "df383424-5b3c-4278-a7e0-7acc933c82d4" (UID: "df383424-5b3c-4278-a7e0-7acc933c82d4"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:36:03.783221 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.783114 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "df383424-5b3c-4278-a7e0-7acc933c82d4" (UID: "df383424-5b3c-4278-a7e0-7acc933c82d4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:36:03.783221 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.783143 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "df383424-5b3c-4278-a7e0-7acc933c82d4" (UID: "df383424-5b3c-4278-a7e0-7acc933c82d4"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:36:03.783403 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.783336 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df383424-5b3c-4278-a7e0-7acc933c82d4-kube-api-access-cmlwq" (OuterVolumeSpecName: "kube-api-access-cmlwq") pod "df383424-5b3c-4278-a7e0-7acc933c82d4" (UID: "df383424-5b3c-4278-a7e0-7acc933c82d4"). InnerVolumeSpecName "kube-api-access-cmlwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:36:03.783403 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.783360 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-online-tls" (OuterVolumeSpecName: "online-tls") pod "df383424-5b3c-4278-a7e0-7acc933c82d4" (UID: "df383424-5b3c-4278-a7e0-7acc933c82d4"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:36:03.881775 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.881730 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/df383424-5b3c-4278-a7e0-7acc933c82d4-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:36:03.881913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.881783 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:36:03.881913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.881792 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmlwq\" (UniqueName: \"kubernetes.io/projected/df383424-5b3c-4278-a7e0-7acc933c82d4-kube-api-access-cmlwq\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:36:03.881913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.881802 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:36:03.881913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.881811 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:36:03.881913 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:03.881819 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/df383424-5b3c-4278-a7e0-7acc933c82d4-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:36:04.174892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.174812 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" event={"ID":"df383424-5b3c-4278-a7e0-7acc933c82d4","Type":"ContainerDied","Data":"6e8a93d268b9b4c33807d00413d0e80785d295abcf1036366339d7c8353b99c5"} Sep 17 13:36:04.174892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.174848 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r" Sep 17 13:36:04.174892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.174867 2575 scope.go:117] "RemoveContainer" containerID="b2513dea7830b2ab1b9c39c18781c47a876f3fde93a3e023ecb27db803d5c876" Sep 17 13:36:04.183838 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.183818 2575 scope.go:117] "RemoveContainer" containerID="6f61e55b85db790bfee7220ce6eef389628390c7e23659d086f11bdd1c0730d5" Sep 17 13:36:04.192138 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.192122 2575 scope.go:117] "RemoveContainer" containerID="933b14d95d14e52ceb48cd2e55ea7070ef35c7c9ec6af1a827260dd9fec08b8c" Sep 17 13:36:04.199550 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.199520 2575 scope.go:117] "RemoveContainer" containerID="50c7b6a2d8544de2741edce87b3e16aac33f7afccff8eece22cc173ec3b756c2" Sep 17 13:36:04.205032 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.205009 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r"] Sep 17 13:36:04.207587 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.207570 2575 scope.go:117] "RemoveContainer" containerID="ca266f741fd497b774006432a4dfeeca14800d2dc459591229ad045c0a250df1" Sep 17 13:36:04.215148 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.215130 2575 scope.go:117] "RemoveContainer" containerID="86e40b1eddd247d6252f882d2fb8306aca8a375350a256d5bd9cb09ce7f85759" Sep 17 13:36:04.215849 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:04.215828 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-dh42r"] Sep 17 13:36:05.607636 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:05.607607 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" path="/var/lib/kubelet/pods/df383424-5b3c-4278-a7e0-7acc933c82d4/volumes" Sep 17 13:36:58.663478 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663440 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6"] Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663769 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="offline" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663781 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="offline" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663792 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="feast-apply" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663798 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="feast-apply" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663814 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="feast-init" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663820 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="feast-init" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663826 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663835 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663850 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="registry" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663856 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="registry" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663862 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663867 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663916 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="registry" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663928 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="ui" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663934 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="online" Sep 17 13:36:58.663952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.663940 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="df383424-5b3c-4278-a7e0-7acc933c82d4" containerName="offline" Sep 17 13:36:58.668496 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.668476 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.670506 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.670481 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 13:36:58.670648 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.670558 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 13:36:58.670648 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.670572 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 13:36:58.671075 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.671053 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 13:36:58.671075 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.671063 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-cfnqg\"" Sep 17 13:36:58.671227 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.671078 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 13:36:58.671227 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.671094 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 13:36:58.678597 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.678577 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6"] Sep 17 13:36:58.807562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.807530 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.807562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.807563 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0aadbe75-3960-4496-8d41-fa9f23bb69df-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.807829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.807590 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxk8j\" (UniqueName: \"kubernetes.io/projected/0aadbe75-3960-4496-8d41-fa9f23bb69df-kube-api-access-mxk8j\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.807829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.807684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.807829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.807721 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.807829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.807778 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0aadbe75-3960-4496-8d41-fa9f23bb69df-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.909101 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.909068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.909101 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.909102 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0aadbe75-3960-4496-8d41-fa9f23bb69df-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.909382 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.909135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxk8j\" (UniqueName: \"kubernetes.io/projected/0aadbe75-3960-4496-8d41-fa9f23bb69df-kube-api-access-mxk8j\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.909382 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.909187 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.909382 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.909211 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.909382 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.909241 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0aadbe75-3960-4496-8d41-fa9f23bb69df-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.909591 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.909573 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0aadbe75-3960-4496-8d41-fa9f23bb69df-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.909865 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.909845 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0aadbe75-3960-4496-8d41-fa9f23bb69df-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.911756 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.911728 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.911920 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.911899 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.911962 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.911918 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.916999 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.916950 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxk8j\" (UniqueName: \"kubernetes.io/projected/0aadbe75-3960-4496-8d41-fa9f23bb69df-kube-api-access-mxk8j\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-prjk6\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:58.977210 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:58.977178 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:36:59.253114 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:59.253093 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6"] Sep 17 13:36:59.255663 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:36:59.255641 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aadbe75_3960_4496_8d41_fa9f23bb69df.slice/crio-f9f536f6abefc2686f5fbb2167d1d908cff628e4220f99824dbdf1d70f3cff07 WatchSource:0}: Error finding container f9f536f6abefc2686f5fbb2167d1d908cff628e4220f99824dbdf1d70f3cff07: Status 404 returned error can't find the container with id f9f536f6abefc2686f5fbb2167d1d908cff628e4220f99824dbdf1d70f3cff07 Sep 17 13:36:59.400539 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:59.400505 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerStarted","Data":"b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da"} Sep 17 13:36:59.400681 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:36:59.400553 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerStarted","Data":"f9f536f6abefc2686f5fbb2167d1d908cff628e4220f99824dbdf1d70f3cff07"} Sep 17 13:37:04.423014 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:04.422978 2575 generic.go:356] "Generic (PLEG): container finished" podID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerID="b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da" exitCode=0 Sep 17 13:37:04.423408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:04.423049 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerDied","Data":"b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da"} Sep 17 13:37:05.428689 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:05.428656 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerStarted","Data":"63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707"} Sep 17 13:37:31.539144 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:31.539114 2575 generic.go:356] "Generic (PLEG): container finished" podID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerID="63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707" exitCode=0 Sep 17 13:37:31.539572 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:31.539182 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerDied","Data":"63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707"} Sep 17 13:37:32.547686 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:32.547642 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerStarted","Data":"bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471"} Sep 17 13:37:32.548059 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:32.547715 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerStarted","Data":"3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd"} Sep 17 13:37:32.548059 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:32.547735 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerStarted","Data":"92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3"} Sep 17 13:37:32.572822 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:32.572766 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podStartSLOduration=34.572747493 podStartE2EDuration="34.572747493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:36:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:37:32.570949357 +0000 UTC m=+1361.475518035" watchObservedRunningTime="2026-09-17 13:37:32.572747493 +0000 UTC m=+1361.477316169" Sep 17 13:37:34.978372 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:34.978335 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:34.978849 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:34.978399 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:34.978849 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:34.978419 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:34.980338 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:34.980303 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused" Sep 17 13:37:34.980475 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:34.980338 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused" Sep 17 13:37:34.980475 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:34.980332 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused" Sep 17 13:37:37.978438 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:37.978394 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused" Sep 17 13:37:37.978438 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:37.978436 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused" Sep 17 13:37:37.978855 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:37.978396 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused" Sep 17 13:37:38.977357 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:38.977317 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:38.977521 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:38.977368 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:38.977521 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:38.977382 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:40.978876 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:40.978847 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:40.979352 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:40.978997 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:40.979743 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:40.979724 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:40.980396 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:40.980376 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:41.982421 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:41.982378 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 13:37:44.886290 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:44.886244 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:37:45.821332 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:37:45.821301 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:38:45.891492 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:45.891397 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 13:38:47.041457 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.041428 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6"] Sep 17 13:38:47.041891 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.041707 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" containerID="cri-o://92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3" gracePeriod=30 Sep 17 13:38:47.041891 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.041802 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" containerID="cri-o://bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471" gracePeriod=30 Sep 17 13:38:47.042019 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.041807 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="offline" containerID="cri-o://3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd" gracePeriod=30 Sep 17 13:38:47.266617 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.266584 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt"] Sep 17 13:38:47.266945 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.266892 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="registry" containerID="cri-o://a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1" gracePeriod=30 Sep 17 13:38:47.267064 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.266933 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" containerID="cri-o://338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e" gracePeriod=30 Sep 17 13:38:47.267064 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.266917 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="offline" containerID="cri-o://3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27" gracePeriod=30 Sep 17 13:38:47.267064 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.266933 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" containerID="cri-o://1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea" gracePeriod=30 Sep 17 13:38:47.860796 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.860765 2575 generic.go:356] "Generic (PLEG): container finished" podID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerID="bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471" exitCode=0 Sep 17 13:38:47.860796 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.860795 2575 generic.go:356] "Generic (PLEG): container finished" podID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerID="92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3" exitCode=0 Sep 17 13:38:47.860938 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.860839 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerDied","Data":"bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471"} Sep 17 13:38:47.860938 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:47.860890 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerDied","Data":"92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3"} Sep 17 13:38:48.868944 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:48.868904 2575 generic.go:356] "Generic (PLEG): container finished" podID="be2d9892-1b19-453c-b273-ac50dae08515" containerID="1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea" exitCode=0 Sep 17 13:38:48.868944 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:48.868935 2575 generic.go:356] "Generic (PLEG): container finished" podID="be2d9892-1b19-453c-b273-ac50dae08515" containerID="338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e" exitCode=0 Sep 17 13:38:48.869353 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:48.868976 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerDied","Data":"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea"} Sep 17 13:38:48.869353 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:48.869015 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerDied","Data":"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e"} Sep 17 13:38:50.979693 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:50.979648 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused" Sep 17 13:38:54.837253 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:54.837206 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 13:38:54.887204 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:54.887167 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused" Sep 17 13:38:55.134847 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:38:55.134764 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 13:39:00.980425 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:00.980378 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused" Sep 17 13:39:04.837472 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:04.837427 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 13:39:04.887240 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:04.887206 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused" Sep 17 13:39:04.887393 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:04.887322 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:39:05.134897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:05.134805 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 13:39:10.980324 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:10.980260 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused" Sep 17 13:39:10.980724 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:10.980416 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:39:14.837438 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:14.837368 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused" Sep 17 13:39:14.837875 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:14.837531 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:39:14.887043 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:14.887002 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused" Sep 17 13:39:15.135771 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:15.135683 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused" Sep 17 13:39:15.135924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:15.135791 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:39:17.681085 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.681062 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:39:17.850681 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.850651 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-ui-tls\") pod \"0aadbe75-3960-4496-8d41-fa9f23bb69df\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " Sep 17 13:39:17.850819 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.850749 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0aadbe75-3960-4496-8d41-fa9f23bb69df-feast-data\") pod \"0aadbe75-3960-4496-8d41-fa9f23bb69df\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " Sep 17 13:39:17.850819 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.850778 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0aadbe75-3960-4496-8d41-fa9f23bb69df-registry-tls\") pod \"0aadbe75-3960-4496-8d41-fa9f23bb69df\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " Sep 17 13:39:17.850931 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.850817 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-online-tls\") pod \"0aadbe75-3960-4496-8d41-fa9f23bb69df\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " Sep 17 13:39:17.850931 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.850881 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-offline-tls\") pod \"0aadbe75-3960-4496-8d41-fa9f23bb69df\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " Sep 17 13:39:17.851035 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.850914 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxk8j\" (UniqueName: \"kubernetes.io/projected/0aadbe75-3960-4496-8d41-fa9f23bb69df-kube-api-access-mxk8j\") pod \"0aadbe75-3960-4496-8d41-fa9f23bb69df\" (UID: \"0aadbe75-3960-4496-8d41-fa9f23bb69df\") " Sep 17 13:39:17.851169 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.851136 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aadbe75-3960-4496-8d41-fa9f23bb69df-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0aadbe75-3960-4496-8d41-fa9f23bb69df" (UID: "0aadbe75-3960-4496-8d41-fa9f23bb69df"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:39:17.851289 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.851241 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aadbe75-3960-4496-8d41-fa9f23bb69df-feast-data" (OuterVolumeSpecName: "feast-data") pod "0aadbe75-3960-4496-8d41-fa9f23bb69df" (UID: "0aadbe75-3960-4496-8d41-fa9f23bb69df"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:39:17.853246 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.853215 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aadbe75-3960-4496-8d41-fa9f23bb69df-kube-api-access-mxk8j" (OuterVolumeSpecName: "kube-api-access-mxk8j") pod "0aadbe75-3960-4496-8d41-fa9f23bb69df" (UID: "0aadbe75-3960-4496-8d41-fa9f23bb69df"). InnerVolumeSpecName "kube-api-access-mxk8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:39:17.853391 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.853359 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-online-tls" (OuterVolumeSpecName: "online-tls") pod "0aadbe75-3960-4496-8d41-fa9f23bb69df" (UID: "0aadbe75-3960-4496-8d41-fa9f23bb69df"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:39:17.853503 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.853448 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0aadbe75-3960-4496-8d41-fa9f23bb69df" (UID: "0aadbe75-3960-4496-8d41-fa9f23bb69df"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:39:17.853566 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.853524 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0aadbe75-3960-4496-8d41-fa9f23bb69df" (UID: "0aadbe75-3960-4496-8d41-fa9f23bb69df"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:39:17.896403 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.896382 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:39:17.951831 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.951808 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0aadbe75-3960-4496-8d41-fa9f23bb69df-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:17.951831 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.951829 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0aadbe75-3960-4496-8d41-fa9f23bb69df-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:17.951997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.951838 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:17.951997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.951845 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:17.951997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.951853 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxk8j\" (UniqueName: \"kubernetes.io/projected/0aadbe75-3960-4496-8d41-fa9f23bb69df-kube-api-access-mxk8j\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:17.951997 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.951862 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0aadbe75-3960-4496-8d41-fa9f23bb69df-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:17.984313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.984259 2575 generic.go:356] "Generic (PLEG): container finished" podID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerID="3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd" exitCode=137 Sep 17 13:39:17.984313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.984298 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerDied","Data":"3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd"} Sep 17 13:39:17.984512 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.984341 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" event={"ID":"0aadbe75-3960-4496-8d41-fa9f23bb69df","Type":"ContainerDied","Data":"f9f536f6abefc2686f5fbb2167d1d908cff628e4220f99824dbdf1d70f3cff07"} Sep 17 13:39:17.984512 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.984358 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6" Sep 17 13:39:17.984512 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.984361 2575 scope.go:117] "RemoveContainer" containerID="bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471" Sep 17 13:39:17.987582 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.987556 2575 generic.go:356] "Generic (PLEG): container finished" podID="be2d9892-1b19-453c-b273-ac50dae08515" containerID="3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27" exitCode=137 Sep 17 13:39:17.987582 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.987584 2575 generic.go:356] "Generic (PLEG): container finished" podID="be2d9892-1b19-453c-b273-ac50dae08515" containerID="a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1" exitCode=137 Sep 17 13:39:17.987738 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.987608 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" Sep 17 13:39:17.987738 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.987610 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerDied","Data":"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27"} Sep 17 13:39:17.987738 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.987647 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerDied","Data":"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1"} Sep 17 13:39:17.987738 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.987669 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt" event={"ID":"be2d9892-1b19-453c-b273-ac50dae08515","Type":"ContainerDied","Data":"e4e8ffa1f0d709b2fc6d113d26e04a865452ace4b167f4486e2b021a2ace8d11"} Sep 17 13:39:17.993744 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:17.993727 2575 scope.go:117] "RemoveContainer" containerID="3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd" Sep 17 13:39:18.001066 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.001051 2575 scope.go:117] "RemoveContainer" containerID="92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3" Sep 17 13:39:18.008753 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.008683 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6"] Sep 17 13:39:18.008806 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.008779 2575 scope.go:117] "RemoveContainer" containerID="63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707" Sep 17 13:39:18.013904 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.013876 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-prjk6"] Sep 17 13:39:18.016002 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.015977 2575 scope.go:117] "RemoveContainer" containerID="b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da" Sep 17 13:39:18.022785 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.022769 2575 scope.go:117] "RemoveContainer" containerID="bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471" Sep 17 13:39:18.023043 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.023026 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471\": container with ID starting with bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471 not found: ID does not exist" containerID="bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471" Sep 17 13:39:18.023087 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.023051 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471"} err="failed to get container status \"bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471\": rpc error: code = NotFound desc = could not find container \"bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471\": container with ID starting with bdbe071f192f9b170ca7eb92c488ed46e750a44cd7cb52e59f54772f5b917471 not found: ID does not exist" Sep 17 13:39:18.023087 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.023067 2575 scope.go:117] "RemoveContainer" containerID="3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd" Sep 17 13:39:18.023318 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.023299 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd\": container with ID starting with 3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd not found: ID does not exist" containerID="3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd" Sep 17 13:39:18.023369 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.023327 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd"} err="failed to get container status \"3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd\": rpc error: code = NotFound desc = could not find container \"3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd\": container with ID starting with 3d389121d1b4ef67a11c46cb7556b46cff4f75ecb712772b3773096f376d1cfd not found: ID does not exist" Sep 17 13:39:18.023369 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.023350 2575 scope.go:117] "RemoveContainer" containerID="92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3" Sep 17 13:39:18.023615 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.023597 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3\": container with ID starting with 92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3 not found: ID does not exist" containerID="92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3" Sep 17 13:39:18.023666 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.023619 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3"} err="failed to get container status \"92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3\": rpc error: code = NotFound desc = could not find container \"92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3\": container with ID starting with 92c60ea8d9cdd84490492ff460cfc196fa22d681a9de1dca4fd81897665d0ff3 not found: ID does not exist" Sep 17 13:39:18.023666 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.023633 2575 scope.go:117] "RemoveContainer" containerID="63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707" Sep 17 13:39:18.023860 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.023845 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707\": container with ID starting with 63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707 not found: ID does not exist" containerID="63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707" Sep 17 13:39:18.023901 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.023865 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707"} err="failed to get container status \"63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707\": rpc error: code = NotFound desc = could not find container \"63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707\": container with ID starting with 63a9ad9416e7599cb414b9ea496f9d4568bff2c087517a9487712adb01cc3707 not found: ID does not exist" Sep 17 13:39:18.023901 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.023881 2575 scope.go:117] "RemoveContainer" containerID="b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da" Sep 17 13:39:18.024114 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.024099 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da\": container with ID starting with b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da not found: ID does not exist" containerID="b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da" Sep 17 13:39:18.024152 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.024122 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da"} err="failed to get container status \"b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da\": rpc error: code = NotFound desc = could not find container \"b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da\": container with ID starting with b5b7cdcf48c5eb4076d045a0d73b4f55b75d47a505c6c3c2bfd613cef82a63da not found: ID does not exist" Sep 17 13:39:18.024152 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.024135 2575 scope.go:117] "RemoveContainer" containerID="1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea" Sep 17 13:39:18.030863 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.030849 2575 scope.go:117] "RemoveContainer" containerID="3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27" Sep 17 13:39:18.037616 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.037597 2575 scope.go:117] "RemoveContainer" containerID="338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e" Sep 17 13:39:18.044292 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.044257 2575 scope.go:117] "RemoveContainer" containerID="a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1" Sep 17 13:39:18.052100 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.051891 2575 scope.go:117] "RemoveContainer" containerID="25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117" Sep 17 13:39:18.052100 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.052088 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-offline-tls\") pod \"be2d9892-1b19-453c-b273-ac50dae08515\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " Sep 17 13:39:18.052217 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.052120 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlzgb\" (UniqueName: \"kubernetes.io/projected/be2d9892-1b19-453c-b273-ac50dae08515-kube-api-access-qlzgb\") pod \"be2d9892-1b19-453c-b273-ac50dae08515\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " Sep 17 13:39:18.052217 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.052156 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be2d9892-1b19-453c-b273-ac50dae08515-feast-data\") pod \"be2d9892-1b19-453c-b273-ac50dae08515\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " Sep 17 13:39:18.052217 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.052200 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-registry-tls\") pod \"be2d9892-1b19-453c-b273-ac50dae08515\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " Sep 17 13:39:18.052393 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.052334 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-ui-tls\") pod \"be2d9892-1b19-453c-b273-ac50dae08515\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " Sep 17 13:39:18.052393 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.052362 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-online-tls\") pod \"be2d9892-1b19-453c-b273-ac50dae08515\" (UID: \"be2d9892-1b19-453c-b273-ac50dae08515\") " Sep 17 13:39:18.053323 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.053218 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be2d9892-1b19-453c-b273-ac50dae08515-feast-data" (OuterVolumeSpecName: "feast-data") pod "be2d9892-1b19-453c-b273-ac50dae08515" (UID: "be2d9892-1b19-453c-b273-ac50dae08515"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:39:18.054870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.054643 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2d9892-1b19-453c-b273-ac50dae08515-kube-api-access-qlzgb" (OuterVolumeSpecName: "kube-api-access-qlzgb") pod "be2d9892-1b19-453c-b273-ac50dae08515" (UID: "be2d9892-1b19-453c-b273-ac50dae08515"). InnerVolumeSpecName "kube-api-access-qlzgb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:39:18.054870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.054671 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "be2d9892-1b19-453c-b273-ac50dae08515" (UID: "be2d9892-1b19-453c-b273-ac50dae08515"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:39:18.054870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.054807 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "be2d9892-1b19-453c-b273-ac50dae08515" (UID: "be2d9892-1b19-453c-b273-ac50dae08515"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:39:18.055177 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.055147 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-online-tls" (OuterVolumeSpecName: "online-tls") pod "be2d9892-1b19-453c-b273-ac50dae08515" (UID: "be2d9892-1b19-453c-b273-ac50dae08515"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:39:18.055681 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.055660 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "be2d9892-1b19-453c-b273-ac50dae08515" (UID: "be2d9892-1b19-453c-b273-ac50dae08515"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:39:18.068949 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.068932 2575 scope.go:117] "RemoveContainer" containerID="d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82" Sep 17 13:39:18.075723 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.075706 2575 scope.go:117] "RemoveContainer" containerID="1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea" Sep 17 13:39:18.075975 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.075959 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea\": container with ID starting with 1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea not found: ID does not exist" containerID="1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea" Sep 17 13:39:18.076016 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.075989 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea"} err="failed to get container status \"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea\": rpc error: code = NotFound desc = could not find container \"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea\": container with ID starting with 1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea not found: ID does not exist" Sep 17 13:39:18.076016 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076006 2575 scope.go:117] "RemoveContainer" containerID="3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27" Sep 17 13:39:18.076226 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.076211 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27\": container with ID starting with 3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27 not found: ID does not exist" containerID="3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27" Sep 17 13:39:18.076266 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076234 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27"} err="failed to get container status \"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27\": rpc error: code = NotFound desc = could not find container \"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27\": container with ID starting with 3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27 not found: ID does not exist" Sep 17 13:39:18.076266 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076250 2575 scope.go:117] "RemoveContainer" containerID="338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e" Sep 17 13:39:18.076483 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.076464 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e\": container with ID starting with 338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e not found: ID does not exist" containerID="338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e" Sep 17 13:39:18.076520 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076487 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e"} err="failed to get container status \"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e\": rpc error: code = NotFound desc = could not find container \"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e\": container with ID starting with 338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e not found: ID does not exist" Sep 17 13:39:18.076520 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076498 2575 scope.go:117] "RemoveContainer" containerID="a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1" Sep 17 13:39:18.076728 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.076714 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1\": container with ID starting with a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1 not found: ID does not exist" containerID="a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1" Sep 17 13:39:18.076775 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076732 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1"} err="failed to get container status \"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1\": rpc error: code = NotFound desc = could not find container \"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1\": container with ID starting with a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1 not found: ID does not exist" Sep 17 13:39:18.076775 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076746 2575 scope.go:117] "RemoveContainer" containerID="25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117" Sep 17 13:39:18.076926 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.076910 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117\": container with ID starting with 25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117 not found: ID does not exist" containerID="25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117" Sep 17 13:39:18.076965 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076929 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117"} err="failed to get container status \"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117\": rpc error: code = NotFound desc = could not find container \"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117\": container with ID starting with 25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117 not found: ID does not exist" Sep 17 13:39:18.076965 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.076940 2575 scope.go:117] "RemoveContainer" containerID="d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82" Sep 17 13:39:18.077157 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:18.077142 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82\": container with ID starting with d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82 not found: ID does not exist" containerID="d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82" Sep 17 13:39:18.077194 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.077160 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82"} err="failed to get container status \"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82\": rpc error: code = NotFound desc = could not find container \"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82\": container with ID starting with d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82 not found: ID does not exist" Sep 17 13:39:18.077194 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.077170 2575 scope.go:117] "RemoveContainer" containerID="1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea" Sep 17 13:39:18.077441 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.077421 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea"} err="failed to get container status \"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea\": rpc error: code = NotFound desc = could not find container \"1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea\": container with ID starting with 1b48220dec4a1c27d6c4996664ef0961716da12c58346e6e50a5f4795c4e59ea not found: ID does not exist" Sep 17 13:39:18.077497 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.077449 2575 scope.go:117] "RemoveContainer" containerID="3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27" Sep 17 13:39:18.077675 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.077655 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27"} err="failed to get container status \"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27\": rpc error: code = NotFound desc = could not find container \"3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27\": container with ID starting with 3fbf264f4a148adb031b565dfa04323cdaac985a7e2c79e3be1d3a4ebf780e27 not found: ID does not exist" Sep 17 13:39:18.077719 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.077676 2575 scope.go:117] "RemoveContainer" containerID="338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e" Sep 17 13:39:18.077920 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.077901 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e"} err="failed to get container status \"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e\": rpc error: code = NotFound desc = could not find container \"338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e\": container with ID starting with 338f43a9dddd7ac6d7a0211532180d05fb22ade447a8cb21decc98cf7002ee4e not found: ID does not exist" Sep 17 13:39:18.077998 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.077922 2575 scope.go:117] "RemoveContainer" containerID="a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1" Sep 17 13:39:18.078151 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.078135 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1"} err="failed to get container status \"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1\": rpc error: code = NotFound desc = could not find container \"a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1\": container with ID starting with a098903486363ea1c99e851191008ae0536fa4b2a53349453369f5bf913a61a1 not found: ID does not exist" Sep 17 13:39:18.078205 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.078151 2575 scope.go:117] "RemoveContainer" containerID="25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117" Sep 17 13:39:18.078379 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.078364 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117"} err="failed to get container status \"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117\": rpc error: code = NotFound desc = could not find container \"25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117\": container with ID starting with 25f037461fc1ecc28cb086b8da50fd70e34c1c9da0dbf440c2fe6295d60c5117 not found: ID does not exist" Sep 17 13:39:18.078426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.078379 2575 scope.go:117] "RemoveContainer" containerID="d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82" Sep 17 13:39:18.078592 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.078575 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82"} err="failed to get container status \"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82\": rpc error: code = NotFound desc = could not find container \"d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82\": container with ID starting with d62aa76d8645eb7c34225a2d25573d8f2310d2613ff50a11b189ce873128be82 not found: ID does not exist" Sep 17 13:39:18.153625 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.153555 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:18.153625 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.153584 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlzgb\" (UniqueName: \"kubernetes.io/projected/be2d9892-1b19-453c-b273-ac50dae08515-kube-api-access-qlzgb\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:18.153625 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.153598 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be2d9892-1b19-453c-b273-ac50dae08515-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:18.153625 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.153610 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:18.153625 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.153620 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:18.153840 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.153633 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be2d9892-1b19-453c-b273-ac50dae08515-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:18.313344 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.313316 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt"] Sep 17 13:39:18.322348 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:18.322330 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nbmtt"] Sep 17 13:39:19.607171 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:19.607137 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" path="/var/lib/kubelet/pods/0aadbe75-3960-4496-8d41-fa9f23bb69df/volumes" Sep 17 13:39:19.607693 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:19.607678 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be2d9892-1b19-453c-b273-ac50dae08515" path="/var/lib/kubelet/pods/be2d9892-1b19-453c-b273-ac50dae08515/volumes" Sep 17 13:39:37.695021 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:37.694986 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf"] Sep 17 13:39:37.695507 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:37.695245 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" podUID="176770f7-eb60-42d1-9cc5-b96f66366127" containerName="manager" containerID="cri-o://a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279" gracePeriod=10 Sep 17 13:39:37.928670 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:37.928644 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:39:38.004300 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.004219 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s6jr\" (UniqueName: \"kubernetes.io/projected/176770f7-eb60-42d1-9cc5-b96f66366127-kube-api-access-7s6jr\") pod \"176770f7-eb60-42d1-9cc5-b96f66366127\" (UID: \"176770f7-eb60-42d1-9cc5-b96f66366127\") " Sep 17 13:39:38.006423 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.006398 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/176770f7-eb60-42d1-9cc5-b96f66366127-kube-api-access-7s6jr" (OuterVolumeSpecName: "kube-api-access-7s6jr") pod "176770f7-eb60-42d1-9cc5-b96f66366127" (UID: "176770f7-eb60-42d1-9cc5-b96f66366127"). InnerVolumeSpecName "kube-api-access-7s6jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:39:38.065246 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.065219 2575 generic.go:356] "Generic (PLEG): container finished" podID="176770f7-eb60-42d1-9cc5-b96f66366127" containerID="a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279" exitCode=0 Sep 17 13:39:38.065391 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.065263 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" Sep 17 13:39:38.065391 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.065310 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" event={"ID":"176770f7-eb60-42d1-9cc5-b96f66366127","Type":"ContainerDied","Data":"a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279"} Sep 17 13:39:38.065391 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.065377 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf" event={"ID":"176770f7-eb60-42d1-9cc5-b96f66366127","Type":"ContainerDied","Data":"c852a038fd284ebbd01c8389945ce701f138218b43d1335226bc358c55bd5f92"} Sep 17 13:39:38.065545 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.065408 2575 scope.go:117] "RemoveContainer" containerID="a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279" Sep 17 13:39:38.073465 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.073449 2575 scope.go:117] "RemoveContainer" containerID="a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279" Sep 17 13:39:38.073740 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:38.073710 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279\": container with ID starting with a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279 not found: ID does not exist" containerID="a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279" Sep 17 13:39:38.073804 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.073737 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279"} err="failed to get container status \"a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279\": rpc error: code = NotFound desc = could not find container \"a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279\": container with ID starting with a410fbb0c093c5c69ed8ee548ab66afaa57456b7f07887eadd3b9da511cbe279 not found: ID does not exist" Sep 17 13:39:38.086520 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.086500 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf"] Sep 17 13:39:38.096789 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.096770 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-k6tdf"] Sep 17 13:39:38.105742 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:38.105718 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s6jr\" (UniqueName: \"kubernetes.io/projected/176770f7-eb60-42d1-9cc5-b96f66366127-kube-api-access-7s6jr\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:39:39.606925 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:39.606893 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="176770f7-eb60-42d1-9cc5-b96f66366127" path="/var/lib/kubelet/pods/176770f7-eb60-42d1-9cc5-b96f66366127/volumes" Sep 17 13:39:42.461814 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.461770 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k"] Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462090 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="feast-apply" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462103 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="feast-apply" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462112 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="feast-init" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462117 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="feast-init" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462124 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462129 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462135 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462140 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462147 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="registry" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462152 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="registry" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462157 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462169 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462176 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="176770f7-eb60-42d1-9cc5-b96f66366127" containerName="manager" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462181 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="176770f7-eb60-42d1-9cc5-b96f66366127" containerName="manager" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462187 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" Sep 17 13:39:42.462192 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462192 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462203 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="offline" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462207 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="offline" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462216 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="feast-apply" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462221 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="feast-apply" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462226 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="offline" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462232 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="offline" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462240 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="feast-init" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462244 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="feast-init" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462299 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="ui" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462306 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="offline" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462313 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="ui" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462318 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="offline" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462325 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="online" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462329 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2d9892-1b19-453c-b273-ac50dae08515" containerName="registry" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462333 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aadbe75-3960-4496-8d41-fa9f23bb69df" containerName="online" Sep 17 13:39:42.462672 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.462338 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="176770f7-eb60-42d1-9cc5-b96f66366127" containerName="manager" Sep 17 13:39:42.466509 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.466493 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" Sep 17 13:39:42.468389 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.468369 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 13:39:42.468737 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.468724 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 13:39:42.468784 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.468724 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-dqtft\"" Sep 17 13:39:42.476029 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.476009 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k"] Sep 17 13:39:42.540528 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.540500 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzc7d\" (UniqueName: \"kubernetes.io/projected/c43437b6-075b-4540-ab32-78d5e887a2e4-kube-api-access-xzc7d\") pod \"feast-operator-controller-manager-689b877b4b-sjb6k\" (UID: \"c43437b6-075b-4540-ab32-78d5e887a2e4\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" Sep 17 13:39:42.641057 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.641029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzc7d\" (UniqueName: \"kubernetes.io/projected/c43437b6-075b-4540-ab32-78d5e887a2e4-kube-api-access-xzc7d\") pod \"feast-operator-controller-manager-689b877b4b-sjb6k\" (UID: \"c43437b6-075b-4540-ab32-78d5e887a2e4\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" Sep 17 13:39:42.655567 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.655539 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzc7d\" (UniqueName: \"kubernetes.io/projected/c43437b6-075b-4540-ab32-78d5e887a2e4-kube-api-access-xzc7d\") pod \"feast-operator-controller-manager-689b877b4b-sjb6k\" (UID: \"c43437b6-075b-4540-ab32-78d5e887a2e4\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" Sep 17 13:39:42.776639 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:42.776603 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" Sep 17 13:39:43.055675 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:43.055601 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k"] Sep 17 13:39:43.058554 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:39:43.058528 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43437b6_075b_4540_ab32_78d5e887a2e4.slice/crio-230335f8a6c57c625b78544ae5803cfe5d13e7cf974c71f6fe59d10aea0c9fb1 WatchSource:0}: Error finding container 230335f8a6c57c625b78544ae5803cfe5d13e7cf974c71f6fe59d10aea0c9fb1: Status 404 returned error can't find the container with id 230335f8a6c57c625b78544ae5803cfe5d13e7cf974c71f6fe59d10aea0c9fb1 Sep 17 13:39:43.090752 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:43.090722 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" event={"ID":"c43437b6-075b-4540-ab32-78d5e887a2e4","Type":"ContainerStarted","Data":"230335f8a6c57c625b78544ae5803cfe5d13e7cf974c71f6fe59d10aea0c9fb1"} Sep 17 13:39:44.096538 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:44.096503 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" event={"ID":"c43437b6-075b-4540-ab32-78d5e887a2e4","Type":"ContainerStarted","Data":"cb95dc6ea0315d768c77952fd759b112a4c195cde311d1e80a3a8b80e2ba26ec"} Sep 17 13:39:44.096905 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:44.096571 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" Sep 17 13:39:44.121012 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:44.120972 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" podStartSLOduration=2.120958695 podStartE2EDuration="2.120958695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:39:44.119914195 +0000 UTC m=+1493.024482863" watchObservedRunningTime="2026-09-17 13:39:44.120958695 +0000 UTC m=+1493.025527398" Sep 17 13:39:51.601375 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:51.601345 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:39:51.604407 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:51.604383 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:39:55.102996 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:55.102967 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-sjb6k" Sep 17 13:39:59.805956 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.805918 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6"] Sep 17 13:39:59.811184 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.811168 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.813313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.813286 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 13:39:59.813313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.813314 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 13:39:59.813459 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.813316 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-gp9tt\"" Sep 17 13:39:59.813459 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.813296 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 13:39:59.813581 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.813565 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 13:39:59.813622 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.813615 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 13:39:59.813664 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.813647 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 13:39:59.825589 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.825567 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6"] Sep 17 13:39:59.878918 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.878889 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxx9c\" (UniqueName: \"kubernetes.io/projected/466422f7-efc3-43ae-8e25-0d063436d2ea-kube-api-access-sxx9c\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.879090 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.878934 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/466422f7-efc3-43ae-8e25-0d063436d2ea-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.879090 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.879025 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.879090 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.879072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.879220 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.879096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.879220 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.879111 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.980363 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.980329 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.980546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.980378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.980546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.980410 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.980546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.980429 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.980546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.980472 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx9c\" (UniqueName: \"kubernetes.io/projected/466422f7-efc3-43ae-8e25-0d063436d2ea-kube-api-access-sxx9c\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.980546 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:59.980473 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 17 13:39:59.980805 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:59.980562 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 17 13:39:59.980805 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.980505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/466422f7-efc3-43ae-8e25-0d063436d2ea-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.980805 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:59.980597 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls podName:466422f7-efc3-43ae-8e25-0d063436d2ea nodeName:}" failed. No retries permitted until 2026-09-17 13:40:00.480577732 +0000 UTC m=+1509.385146388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls") pod "feast-simple-feast-setup-57576b8fcf-zv6h6" (UID: "466422f7-efc3-43ae-8e25-0d063436d2ea") : secret "feast-simple-feast-setup-online-tls" not found Sep 17 13:39:59.980805 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:39:59.980624 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls podName:466422f7-efc3-43ae-8e25-0d063436d2ea nodeName:}" failed. No retries permitted until 2026-09-17 13:40:00.480616642 +0000 UTC m=+1509.385185297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-zv6h6" (UID: "466422f7-efc3-43ae-8e25-0d063436d2ea") : secret "feast-simple-feast-setup-registry-tls" not found Sep 17 13:39:59.980981 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.980865 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/466422f7-efc3-43ae-8e25-0d063436d2ea-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.982901 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.982878 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.983006 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.982990 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:39:59.990817 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:39:59.990799 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxx9c\" (UniqueName: \"kubernetes.io/projected/466422f7-efc3-43ae-8e25-0d063436d2ea-kube-api-access-sxx9c\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:00.485786 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:00.485755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:00.485955 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:00.485821 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:00.488308 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:00.488284 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:00.488431 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:00.488309 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zv6h6\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:00.729237 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:00.729200 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:01.010047 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:01.010016 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6"] Sep 17 13:40:01.013083 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:40:01.013049 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod466422f7_efc3_43ae_8e25_0d063436d2ea.slice/crio-9224bb673c1582ec6617bba31aeb8c89be25853806f811c6798311f6fcbeee5e WatchSource:0}: Error finding container 9224bb673c1582ec6617bba31aeb8c89be25853806f811c6798311f6fcbeee5e: Status 404 returned error can't find the container with id 9224bb673c1582ec6617bba31aeb8c89be25853806f811c6798311f6fcbeee5e Sep 17 13:40:01.163548 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:01.163512 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerStarted","Data":"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1"} Sep 17 13:40:01.163708 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:01.163555 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerStarted","Data":"9224bb673c1582ec6617bba31aeb8c89be25853806f811c6798311f6fcbeee5e"} Sep 17 13:40:06.186214 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:06.186127 2575 generic.go:356] "Generic (PLEG): container finished" podID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerID="bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1" exitCode=0 Sep 17 13:40:06.186214 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:06.186191 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerDied","Data":"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1"} Sep 17 13:40:07.192617 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:07.192579 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerStarted","Data":"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2"} Sep 17 13:40:12.214441 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:12.214360 2575 generic.go:356] "Generic (PLEG): container finished" podID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerID="6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2" exitCode=0 Sep 17 13:40:12.214441 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:12.214427 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerDied","Data":"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2"} Sep 17 13:40:13.224408 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:13.224360 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerStarted","Data":"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7"} Sep 17 13:40:13.224867 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:13.224427 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerStarted","Data":"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8"} Sep 17 13:40:13.224867 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:13.224449 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerStarted","Data":"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45"} Sep 17 13:40:13.224867 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:13.224468 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerStarted","Data":"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144"} Sep 17 13:40:13.261833 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:13.261767 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podStartSLOduration=14.261746555 podStartE2EDuration="14.261746555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:40:13.26095771 +0000 UTC m=+1522.165526389" watchObservedRunningTime="2026-09-17 13:40:13.261746555 +0000 UTC m=+1522.166315233" Sep 17 13:40:15.730103 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:15.729999 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:15.730103 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:15.730076 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:15.730103 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:15.730096 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:15.730804 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:15.730781 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:15.732054 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:15.731909 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.40:6571: connect: connection refused" Sep 17 13:40:15.732054 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:15.731921 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused" Sep 17 13:40:15.732054 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:15.731967 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused" Sep 17 13:40:15.732054 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:15.731976 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.40:8816: connect: connection refused" Sep 17 13:40:18.730567 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:18.730038 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused" Sep 17 13:40:18.730567 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:18.730064 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused" Sep 17 13:40:18.730567 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:18.730371 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.40:8816: connect: connection refused" Sep 17 13:40:18.731153 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:18.730649 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.40:6571: connect: connection refused" Sep 17 13:40:20.729747 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:20.729706 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:20.730243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:20.729772 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:20.730243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:20.729793 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:20.730243 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:20.729807 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:21.731424 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:21.731392 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:21.731807 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:21.731528 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:21.731807 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:21.731672 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:21.732428 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:21.732403 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:21.732562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:21.732473 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:21.732562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:21.732520 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:21.736595 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:21.736575 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:40:22.278789 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:40:22.278761 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:41:23.634379 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:23.634341 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6"] Sep 17 13:41:23.634888 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:23.634615 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="registry" containerID="cri-o://bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144" gracePeriod=30 Sep 17 13:41:23.634888 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:23.634702 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" containerID="cri-o://e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7" gracePeriod=30 Sep 17 13:41:23.634888 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:23.634714 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" containerID="cri-o://8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45" gracePeriod=30 Sep 17 13:41:23.634888 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:23.634713 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="offline" containerID="cri-o://1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8" gracePeriod=30 Sep 17 13:41:24.233168 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.233131 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr"] Sep 17 13:41:24.236874 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.236853 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.239751 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.239727 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-x9n82\"" Sep 17 13:41:24.248979 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.248956 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr"] Sep 17 13:41:24.351978 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.351944 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.351978 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.351982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.352194 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.352002 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d65749c2-cc9b-4ae1-aaa1-01b890f16076-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.352194 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.352031 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzklh\" (UniqueName: \"kubernetes.io/projected/d65749c2-cc9b-4ae1-aaa1-01b890f16076-kube-api-access-mzklh\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.352194 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.352169 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.352307 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.352228 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.453330 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.453308 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.453426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.453338 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.453426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.453357 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d65749c2-cc9b-4ae1-aaa1-01b890f16076-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.453426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.453375 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzklh\" (UniqueName: \"kubernetes.io/projected/d65749c2-cc9b-4ae1-aaa1-01b890f16076-kube-api-access-mzklh\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.453426 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.453423 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.453627 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.453445 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.453749 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.453727 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d65749c2-cc9b-4ae1-aaa1-01b890f16076-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.455758 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.455732 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.455907 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.455890 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.455947 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.455914 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.455985 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.455910 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.467335 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.467308 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzklh\" (UniqueName: \"kubernetes.io/projected/d65749c2-cc9b-4ae1-aaa1-01b890f16076-kube-api-access-mzklh\") pod \"feast-simple-feast-setup-57576b8fcf-ctzhr\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.527124 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.527091 2575 generic.go:356] "Generic (PLEG): container finished" podID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerID="e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7" exitCode=0 Sep 17 13:41:24.527124 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.527115 2575 generic.go:356] "Generic (PLEG): container finished" podID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerID="8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45" exitCode=0 Sep 17 13:41:24.527293 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.527159 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerDied","Data":"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7"} Sep 17 13:41:24.527293 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.527200 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerDied","Data":"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45"} Sep 17 13:41:24.546875 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.546852 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:24.841251 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:24.841227 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr"] Sep 17 13:41:24.843220 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:41:24.843196 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd65749c2_cc9b_4ae1_aaa1_01b890f16076.slice/crio-9395ca43a3ce45e6f297757195b041f75da5edc87687c1a07c135186ad6f2e53 WatchSource:0}: Error finding container 9395ca43a3ce45e6f297757195b041f75da5edc87687c1a07c135186ad6f2e53: Status 404 returned error can't find the container with id 9395ca43a3ce45e6f297757195b041f75da5edc87687c1a07c135186ad6f2e53 Sep 17 13:41:25.533053 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:25.533014 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerStarted","Data":"0106d072cb313058d91451b18f0698fdf27bd9b44626655496e9280835b98022"} Sep 17 13:41:25.533225 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:25.533069 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerStarted","Data":"9395ca43a3ce45e6f297757195b041f75da5edc87687c1a07c135186ad6f2e53"} Sep 17 13:41:29.550309 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:29.550258 2575 generic.go:356] "Generic (PLEG): container finished" podID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerID="0106d072cb313058d91451b18f0698fdf27bd9b44626655496e9280835b98022" exitCode=0 Sep 17 13:41:29.550688 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:29.550305 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerDied","Data":"0106d072cb313058d91451b18f0698fdf27bd9b44626655496e9280835b98022"} Sep 17 13:41:30.557437 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:30.557393 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerStarted","Data":"fc116fe9c85dec12fd372f39645c0af1525219fa2af21790c95a6786b7e34f53"} Sep 17 13:41:31.732487 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:31.732435 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused" Sep 17 13:41:32.276739 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:32.276691 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused" Sep 17 13:41:35.579685 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:35.579652 2575 generic.go:356] "Generic (PLEG): container finished" podID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerID="fc116fe9c85dec12fd372f39645c0af1525219fa2af21790c95a6786b7e34f53" exitCode=0 Sep 17 13:41:35.580062 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:35.579696 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerDied","Data":"fc116fe9c85dec12fd372f39645c0af1525219fa2af21790c95a6786b7e34f53"} Sep 17 13:41:36.589759 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:36.589716 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerStarted","Data":"c74b49e1282562e3d2a744e36760e5dde2be70bfb2b80c55a5601c06b43f1273"} Sep 17 13:41:36.590229 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:36.589782 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerStarted","Data":"01a2d0710c7a018c8413cd5dc754e161caadd77b92784d6eea6a60ed7099483e"} Sep 17 13:41:36.590229 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:36.589803 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerStarted","Data":"dbd8ac38b0f0058ba50f172f4a99540afe819663384bf375041b43b58164e375"} Sep 17 13:41:36.590229 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:36.589835 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerStarted","Data":"1e389c66c579121846259a62cc4ad825b8b654172874846648225d7936cd0b79"} Sep 17 13:41:36.622059 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:36.621988 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podStartSLOduration=12.621965792 podStartE2EDuration="12.621965792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:41:36.621795191 +0000 UTC m=+1605.526363871" watchObservedRunningTime="2026-09-17 13:41:36.621965792 +0000 UTC m=+1605.526534469" Sep 17 13:41:39.547492 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:39.547460 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:39.548074 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:39.548049 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:39.548074 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:39.548080 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:39.548308 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:39.548097 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:39.549494 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:39.549457 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.41:6571: connect: connection refused" Sep 17 13:41:39.549624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:39.549483 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused" Sep 17 13:41:39.549624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:39.549494 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.41:8816: connect: connection refused" Sep 17 13:41:39.549624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:39.549549 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused" Sep 17 13:41:41.733322 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:41.733229 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused" Sep 17 13:41:42.276567 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:42.276516 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused" Sep 17 13:41:42.547761 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:42.547506 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.41:8816: connect: connection refused" Sep 17 13:41:42.547761 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:42.547597 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused" Sep 17 13:41:42.547761 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:42.547506 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.41:6571: connect: connection refused" Sep 17 13:41:42.547761 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:42.547679 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused" Sep 17 13:41:44.547314 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:44.547258 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:44.547673 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:44.547333 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:44.547673 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:44.547352 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:44.547673 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:44.547363 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:45.548619 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:45.548585 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:45.549048 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:45.548740 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:45.549048 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:45.548932 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:45.549447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:45.549416 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:45.549624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:45.549606 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:45.549786 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:45.549770 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:45.554080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:45.554060 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:45.642952 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:45.642919 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:41:51.733089 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:51.733046 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused" Sep 17 13:41:51.733535 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:51.733179 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:41:52.276416 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:52.276378 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused" Sep 17 13:41:52.276586 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:52.276496 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:41:54.270033 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.270009 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:41:54.304863 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.304831 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxx9c\" (UniqueName: \"kubernetes.io/projected/466422f7-efc3-43ae-8e25-0d063436d2ea-kube-api-access-sxx9c\") pod \"466422f7-efc3-43ae-8e25-0d063436d2ea\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " Sep 17 13:41:54.304863 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.304862 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls\") pod \"466422f7-efc3-43ae-8e25-0d063436d2ea\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " Sep 17 13:41:54.305088 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.304893 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/466422f7-efc3-43ae-8e25-0d063436d2ea-feast-data\") pod \"466422f7-efc3-43ae-8e25-0d063436d2ea\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " Sep 17 13:41:54.305088 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.305017 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls\") pod \"466422f7-efc3-43ae-8e25-0d063436d2ea\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " Sep 17 13:41:54.305088 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.305060 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-ui-tls\") pod \"466422f7-efc3-43ae-8e25-0d063436d2ea\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " Sep 17 13:41:54.305247 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.305139 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-offline-tls\") pod \"466422f7-efc3-43ae-8e25-0d063436d2ea\" (UID: \"466422f7-efc3-43ae-8e25-0d063436d2ea\") " Sep 17 13:41:54.305924 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.305871 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466422f7-efc3-43ae-8e25-0d063436d2ea-feast-data" (OuterVolumeSpecName: "feast-data") pod "466422f7-efc3-43ae-8e25-0d063436d2ea" (UID: "466422f7-efc3-43ae-8e25-0d063436d2ea"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:41:54.307661 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.307626 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/466422f7-efc3-43ae-8e25-0d063436d2ea-kube-api-access-sxx9c" (OuterVolumeSpecName: "kube-api-access-sxx9c") pod "466422f7-efc3-43ae-8e25-0d063436d2ea" (UID: "466422f7-efc3-43ae-8e25-0d063436d2ea"). InnerVolumeSpecName "kube-api-access-sxx9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:41:54.307892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.307856 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls" (OuterVolumeSpecName: "online-tls") pod "466422f7-efc3-43ae-8e25-0d063436d2ea" (UID: "466422f7-efc3-43ae-8e25-0d063436d2ea"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:41:54.308001 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.307899 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "466422f7-efc3-43ae-8e25-0d063436d2ea" (UID: "466422f7-efc3-43ae-8e25-0d063436d2ea"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:41:54.308001 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.307992 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "466422f7-efc3-43ae-8e25-0d063436d2ea" (UID: "466422f7-efc3-43ae-8e25-0d063436d2ea"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:41:54.308125 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.308018 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "466422f7-efc3-43ae-8e25-0d063436d2ea" (UID: "466422f7-efc3-43ae-8e25-0d063436d2ea"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:41:54.406825 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.406735 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:41:54.406825 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.406772 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxx9c\" (UniqueName: \"kubernetes.io/projected/466422f7-efc3-43ae-8e25-0d063436d2ea-kube-api-access-sxx9c\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:41:54.406825 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.406783 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:41:54.406825 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.406793 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/466422f7-efc3-43ae-8e25-0d063436d2ea-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:41:54.406825 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.406800 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:41:54.406825 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.406807 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/466422f7-efc3-43ae-8e25-0d063436d2ea-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:41:54.676801 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.676723 2575 generic.go:356] "Generic (PLEG): container finished" podID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerID="1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8" exitCode=137 Sep 17 13:41:54.676801 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.676746 2575 generic.go:356] "Generic (PLEG): container finished" podID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerID="bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144" exitCode=137 Sep 17 13:41:54.676801 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.676776 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerDied","Data":"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8"} Sep 17 13:41:54.676801 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.676792 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" Sep 17 13:41:54.677106 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.676808 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerDied","Data":"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144"} Sep 17 13:41:54.677106 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.676822 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6" event={"ID":"466422f7-efc3-43ae-8e25-0d063436d2ea","Type":"ContainerDied","Data":"9224bb673c1582ec6617bba31aeb8c89be25853806f811c6798311f6fcbeee5e"} Sep 17 13:41:54.677106 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.676840 2575 scope.go:117] "RemoveContainer" containerID="e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7" Sep 17 13:41:54.686017 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.686000 2575 scope.go:117] "RemoveContainer" containerID="1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8" Sep 17 13:41:54.693901 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.693885 2575 scope.go:117] "RemoveContainer" containerID="8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45" Sep 17 13:41:54.700897 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.700882 2575 scope.go:117] "RemoveContainer" containerID="bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144" Sep 17 13:41:54.709492 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.709262 2575 scope.go:117] "RemoveContainer" containerID="6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2" Sep 17 13:41:54.710018 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.709976 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6"] Sep 17 13:41:54.715362 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.715341 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zv6h6"] Sep 17 13:41:54.719060 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.719044 2575 scope.go:117] "RemoveContainer" containerID="bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1" Sep 17 13:41:54.726138 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.726122 2575 scope.go:117] "RemoveContainer" containerID="e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7" Sep 17 13:41:54.726425 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:41:54.726408 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7\": container with ID starting with e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7 not found: ID does not exist" containerID="e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7" Sep 17 13:41:54.726490 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.726434 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7"} err="failed to get container status \"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7\": rpc error: code = NotFound desc = could not find container \"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7\": container with ID starting with e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7 not found: ID does not exist" Sep 17 13:41:54.726490 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.726452 2575 scope.go:117] "RemoveContainer" containerID="1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8" Sep 17 13:41:54.726738 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:41:54.726724 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8\": container with ID starting with 1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8 not found: ID does not exist" containerID="1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8" Sep 17 13:41:54.726790 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.726744 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8"} err="failed to get container status \"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8\": rpc error: code = NotFound desc = could not find container \"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8\": container with ID starting with 1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8 not found: ID does not exist" Sep 17 13:41:54.726790 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.726755 2575 scope.go:117] "RemoveContainer" containerID="8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45" Sep 17 13:41:54.726972 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:41:54.726958 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45\": container with ID starting with 8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45 not found: ID does not exist" containerID="8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45" Sep 17 13:41:54.727015 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.726976 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45"} err="failed to get container status \"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45\": rpc error: code = NotFound desc = could not find container \"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45\": container with ID starting with 8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45 not found: ID does not exist" Sep 17 13:41:54.727015 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.726986 2575 scope.go:117] "RemoveContainer" containerID="bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144" Sep 17 13:41:54.727246 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:41:54.727231 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144\": container with ID starting with bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144 not found: ID does not exist" containerID="bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144" Sep 17 13:41:54.727363 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.727248 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144"} err="failed to get container status \"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144\": rpc error: code = NotFound desc = could not find container \"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144\": container with ID starting with bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144 not found: ID does not exist" Sep 17 13:41:54.727363 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.727259 2575 scope.go:117] "RemoveContainer" containerID="6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2" Sep 17 13:41:54.727529 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:41:54.727505 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2\": container with ID starting with 6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2 not found: ID does not exist" containerID="6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2" Sep 17 13:41:54.727582 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.727538 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2"} err="failed to get container status \"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2\": rpc error: code = NotFound desc = could not find container \"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2\": container with ID starting with 6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2 not found: ID does not exist" Sep 17 13:41:54.727582 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.727561 2575 scope.go:117] "RemoveContainer" containerID="bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1" Sep 17 13:41:54.727868 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:41:54.727845 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1\": container with ID starting with bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1 not found: ID does not exist" containerID="bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1" Sep 17 13:41:54.727930 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.727875 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1"} err="failed to get container status \"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1\": rpc error: code = NotFound desc = could not find container \"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1\": container with ID starting with bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1 not found: ID does not exist" Sep 17 13:41:54.727930 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.727896 2575 scope.go:117] "RemoveContainer" containerID="e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7" Sep 17 13:41:54.728152 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.728133 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7"} err="failed to get container status \"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7\": rpc error: code = NotFound desc = could not find container \"e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7\": container with ID starting with e407bd23323041a9c855e5749f5e65cd69d18c5787ff644f09463beaf11b9ed7 not found: ID does not exist" Sep 17 13:41:54.728204 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.728154 2575 scope.go:117] "RemoveContainer" containerID="1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8" Sep 17 13:41:54.728410 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.728392 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8"} err="failed to get container status \"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8\": rpc error: code = NotFound desc = could not find container \"1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8\": container with ID starting with 1ecf6de50b85f0b614c4e847cbbaf1b79a5963a060112cae1f49ad0778915fc8 not found: ID does not exist" Sep 17 13:41:54.728479 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.728411 2575 scope.go:117] "RemoveContainer" containerID="8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45" Sep 17 13:41:54.728639 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.728621 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45"} err="failed to get container status \"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45\": rpc error: code = NotFound desc = could not find container \"8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45\": container with ID starting with 8eaa670f967c8ebd3ff6f95328d3868a35d5fa7cda9ac3d2ea3351db5e301b45 not found: ID does not exist" Sep 17 13:41:54.728639 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.728638 2575 scope.go:117] "RemoveContainer" containerID="bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144" Sep 17 13:41:54.728870 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.728855 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144"} err="failed to get container status \"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144\": rpc error: code = NotFound desc = could not find container \"bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144\": container with ID starting with bf0870790f7fab92f3566e1b7050e6bcb0ac9bce76581058110dddf1be076144 not found: ID does not exist" Sep 17 13:41:54.728919 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.728870 2575 scope.go:117] "RemoveContainer" containerID="6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2" Sep 17 13:41:54.729136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.729114 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2"} err="failed to get container status \"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2\": rpc error: code = NotFound desc = could not find container \"6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2\": container with ID starting with 6ebabdf18056956719bf6240fed5bdb1ae70ed099c34a863290e794f095757c2 not found: ID does not exist" Sep 17 13:41:54.729136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.729135 2575 scope.go:117] "RemoveContainer" containerID="bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1" Sep 17 13:41:54.729411 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:54.729385 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1"} err="failed to get container status \"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1\": rpc error: code = NotFound desc = could not find container \"bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1\": container with ID starting with bd29a93d0f09378fb2efdb1c93f3abca372383d3867003a07ae2dafaad3a11e1 not found: ID does not exist" Sep 17 13:41:55.607831 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:41:55.607802 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" path="/var/lib/kubelet/pods/466422f7-efc3-43ae-8e25-0d063436d2ea/volumes" Sep 17 13:42:48.795757 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.795721 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s"] Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796031 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="feast-init" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796042 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="feast-init" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796049 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="offline" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796055 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="offline" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796068 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="feast-apply" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796073 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="feast-apply" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796087 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796091 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796100 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="registry" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796105 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="registry" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796111 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796115 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796164 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="registry" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796173 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="offline" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796178 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="online" Sep 17 13:42:48.796208 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.796184 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="466422f7-efc3-43ae-8e25-0d063436d2ea" containerName="ui" Sep 17 13:42:48.799768 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.799752 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.801821 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.801793 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 13:42:48.801983 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.801874 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 13:42:48.802064 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.801994 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 13:42:48.802064 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.802037 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 13:42:48.802311 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.802295 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 13:42:48.802311 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.802306 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-wzmd5\"" Sep 17 13:42:48.802452 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.802363 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 13:42:48.810422 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.810403 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s"] Sep 17 13:42:48.853114 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.853090 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.853114 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.853114 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p24b8\" (UniqueName: \"kubernetes.io/projected/7aa386de-5399-4a6b-a4e7-1f015c4d547e-kube-api-access-p24b8\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.853294 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.853137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.853294 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.853160 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7aa386de-5399-4a6b-a4e7-1f015c4d547e-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.853294 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.853259 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.853415 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.853303 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7aa386de-5399-4a6b-a4e7-1f015c4d547e-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.954335 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.954301 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.954335 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.954342 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7aa386de-5399-4a6b-a4e7-1f015c4d547e-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.954593 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.954383 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.954593 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.954399 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p24b8\" (UniqueName: \"kubernetes.io/projected/7aa386de-5399-4a6b-a4e7-1f015c4d547e-kube-api-access-p24b8\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.954593 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.954419 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.954593 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.954441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7aa386de-5399-4a6b-a4e7-1f015c4d547e-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.954593 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:42:48.954477 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 13:42:48.954593 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:42:48.954539 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 13:42:48.954593 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:42:48.954587 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls podName:7aa386de-5399-4a6b-a4e7-1f015c4d547e nodeName:}" failed. No retries permitted until 2026-09-17 13:42:49.454566853 +0000 UTC m=+1678.359135509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" (UID: "7aa386de-5399-4a6b-a4e7-1f015c4d547e") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 13:42:48.954865 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:42:48.954636 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls podName:7aa386de-5399-4a6b-a4e7-1f015c4d547e nodeName:}" failed. No retries permitted until 2026-09-17 13:42:49.454618508 +0000 UTC m=+1678.359187186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" (UID: "7aa386de-5399-4a6b-a4e7-1f015c4d547e") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 13:42:48.954865 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.954774 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7aa386de-5399-4a6b-a4e7-1f015c4d547e-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.955112 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.955097 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7aa386de-5399-4a6b-a4e7-1f015c4d547e-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.957033 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.957014 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:48.970001 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:48.969974 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p24b8\" (UniqueName: \"kubernetes.io/projected/7aa386de-5399-4a6b-a4e7-1f015c4d547e-kube-api-access-p24b8\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:49.458819 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:49.458783 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:49.458984 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:49.458840 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:49.461370 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:49.461351 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:49.461447 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:49.461394 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-5bb4s\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:49.709546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:49.709460 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:42:49.986891 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:49.986870 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s"] Sep 17 13:42:49.988827 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:42:49.988799 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7aa386de_5399_4a6b_a4e7_1f015c4d547e.slice/crio-cf5671f0bfa1066cd4eb4b382a928ded9a899a717e0a55dd81ae32bab883621a WatchSource:0}: Error finding container cf5671f0bfa1066cd4eb4b382a928ded9a899a717e0a55dd81ae32bab883621a: Status 404 returned error can't find the container with id cf5671f0bfa1066cd4eb4b382a928ded9a899a717e0a55dd81ae32bab883621a Sep 17 13:42:50.901777 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:50.901738 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerStarted","Data":"58ccb675a104136b733f22ff6198623a0b423d5d339490e190d280c3732b2760"} Sep 17 13:42:50.901938 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:50.901794 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerStarted","Data":"cf5671f0bfa1066cd4eb4b382a928ded9a899a717e0a55dd81ae32bab883621a"} Sep 17 13:42:54.919163 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:54.919129 2575 generic.go:356] "Generic (PLEG): container finished" podID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerID="58ccb675a104136b733f22ff6198623a0b423d5d339490e190d280c3732b2760" exitCode=0 Sep 17 13:42:54.919601 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:54.919204 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerDied","Data":"58ccb675a104136b733f22ff6198623a0b423d5d339490e190d280c3732b2760"} Sep 17 13:42:55.925328 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:42:55.925290 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerStarted","Data":"d15e68d303433a0ec8af5fedd47eac3c158c5cc1dec64b338b2a3821f2e7a36e"} Sep 17 13:43:20.023989 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:20.023953 2575 generic.go:356] "Generic (PLEG): container finished" podID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerID="d15e68d303433a0ec8af5fedd47eac3c158c5cc1dec64b338b2a3821f2e7a36e" exitCode=0 Sep 17 13:43:20.024537 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:20.024012 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerDied","Data":"d15e68d303433a0ec8af5fedd47eac3c158c5cc1dec64b338b2a3821f2e7a36e"} Sep 17 13:43:21.033554 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:21.033515 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerStarted","Data":"d61b31fe40ca6cda3e609c29d870c5686d4b71d6c5354af3e12623e64c02fb3f"} Sep 17 13:43:21.033915 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:21.033581 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerStarted","Data":"898085082fc4e50c94e771a31e74e1e2d5b814fcd56baccc7e2382299d47df7d"} Sep 17 13:43:21.033915 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:21.033604 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerStarted","Data":"fbe597d4f9a72732911586656c824570042f1992b831800f730fc54ee740e013"} Sep 17 13:43:21.058144 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:21.058075 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podStartSLOduration=33.058053806 podStartE2EDuration="33.058053806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:43:21.055443048 +0000 UTC m=+1709.960011751" watchObservedRunningTime="2026-09-17 13:43:21.058053806 +0000 UTC m=+1709.962622484" Sep 17 13:43:22.710069 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:22.710024 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:22.710564 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:22.710087 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:22.710564 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:22.710105 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:22.711730 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:22.711697 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.42:8816: connect: connection refused" Sep 17 13:43:22.711853 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:22.711729 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused" Sep 17 13:43:22.711853 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:22.711741 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused" Sep 17 13:43:25.710624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:25.710571 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.42:8816: connect: connection refused" Sep 17 13:43:25.710624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:25.710612 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused" Sep 17 13:43:25.711149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:25.710571 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused" Sep 17 13:43:28.711332 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:28.711297 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:28.711754 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:28.711414 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:28.711754 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:28.711695 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:28.711754 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:28.711732 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:28.712077 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:28.712060 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:28.712154 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:28.712139 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:29.710357 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:29.710323 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:29.710543 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:29.710428 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 13:43:32.590868 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:32.590838 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:43:33.573624 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:43:33.573588 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:44:10.715398 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:10.715358 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 17 13:44:30.715415 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:30.715367 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 17 13:44:35.010840 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.010804 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s"] Sep 17 13:44:35.011245 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.011079 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" containerID="cri-o://fbe597d4f9a72732911586656c824570042f1992b831800f730fc54ee740e013" gracePeriod=30 Sep 17 13:44:35.011245 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.011163 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" containerID="cri-o://d61b31fe40ca6cda3e609c29d870c5686d4b71d6c5354af3e12623e64c02fb3f" gracePeriod=30 Sep 17 13:44:35.011413 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.011255 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="offline" containerID="cri-o://898085082fc4e50c94e771a31e74e1e2d5b814fcd56baccc7e2382299d47df7d" gracePeriod=30 Sep 17 13:44:35.207379 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.207346 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr"] Sep 17 13:44:35.207685 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.207634 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="registry" containerID="cri-o://1e389c66c579121846259a62cc4ad825b8b654172874846648225d7936cd0b79" gracePeriod=30 Sep 17 13:44:35.207919 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.207720 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" containerID="cri-o://dbd8ac38b0f0058ba50f172f4a99540afe819663384bf375041b43b58164e375" gracePeriod=30 Sep 17 13:44:35.208028 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.207718 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="offline" containerID="cri-o://01a2d0710c7a018c8413cd5dc754e161caadd77b92784d6eea6a60ed7099483e" gracePeriod=30 Sep 17 13:44:35.208106 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.207748 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" containerID="cri-o://c74b49e1282562e3d2a744e36760e5dde2be70bfb2b80c55a5601c06b43f1273" gracePeriod=30 Sep 17 13:44:35.549871 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.549800 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused" Sep 17 13:44:35.641183 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:35.641141 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused" Sep 17 13:44:36.344749 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:36.344717 2575 generic.go:356] "Generic (PLEG): container finished" podID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerID="c74b49e1282562e3d2a744e36760e5dde2be70bfb2b80c55a5601c06b43f1273" exitCode=0 Sep 17 13:44:36.344749 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:36.344742 2575 generic.go:356] "Generic (PLEG): container finished" podID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerID="dbd8ac38b0f0058ba50f172f4a99540afe819663384bf375041b43b58164e375" exitCode=0 Sep 17 13:44:36.345140 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:36.344785 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerDied","Data":"c74b49e1282562e3d2a744e36760e5dde2be70bfb2b80c55a5601c06b43f1273"} Sep 17 13:44:36.345140 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:36.344827 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerDied","Data":"dbd8ac38b0f0058ba50f172f4a99540afe819663384bf375041b43b58164e375"} Sep 17 13:44:36.347268 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:36.347244 2575 generic.go:356] "Generic (PLEG): container finished" podID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerID="d61b31fe40ca6cda3e609c29d870c5686d4b71d6c5354af3e12623e64c02fb3f" exitCode=0 Sep 17 13:44:36.347268 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:36.347263 2575 generic.go:356] "Generic (PLEG): container finished" podID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerID="fbe597d4f9a72732911586656c824570042f1992b831800f730fc54ee740e013" exitCode=0 Sep 17 13:44:36.347434 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:36.347302 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerDied","Data":"d61b31fe40ca6cda3e609c29d870c5686d4b71d6c5354af3e12623e64c02fb3f"} Sep 17 13:44:36.347434 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:36.347336 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerDied","Data":"fbe597d4f9a72732911586656c824570042f1992b831800f730fc54ee740e013"} Sep 17 13:44:38.712177 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:38.712133 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused" Sep 17 13:44:42.591600 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:42.591516 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused" Sep 17 13:44:45.550003 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:45.549954 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused" Sep 17 13:44:45.640692 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:45.640653 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused" Sep 17 13:44:48.712546 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:48.712498 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused" Sep 17 13:44:51.629088 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:51.629058 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:44:51.633763 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:51.633743 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:44:52.592402 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:52.592356 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused" Sep 17 13:44:55.549640 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:55.549588 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused" Sep 17 13:44:55.550093 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:55.549746 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:44:55.640381 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:55.640342 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused" Sep 17 13:44:55.640556 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:55.640470 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:44:58.712099 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:58.712052 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused" Sep 17 13:44:58.712560 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:44:58.712191 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:45:02.592428 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:02.592387 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused" Sep 17 13:45:02.592792 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:02.592493 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:45:05.469169 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.469067 2575 generic.go:356] "Generic (PLEG): container finished" podID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerID="898085082fc4e50c94e771a31e74e1e2d5b814fcd56baccc7e2382299d47df7d" exitCode=137 Sep 17 13:45:05.469169 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.469149 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerDied","Data":"898085082fc4e50c94e771a31e74e1e2d5b814fcd56baccc7e2382299d47df7d"} Sep 17 13:45:05.472108 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.472085 2575 generic.go:356] "Generic (PLEG): container finished" podID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerID="01a2d0710c7a018c8413cd5dc754e161caadd77b92784d6eea6a60ed7099483e" exitCode=137 Sep 17 13:45:05.472108 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.472104 2575 generic.go:356] "Generic (PLEG): container finished" podID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerID="1e389c66c579121846259a62cc4ad825b8b654172874846648225d7936cd0b79" exitCode=137 Sep 17 13:45:05.472250 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.472148 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerDied","Data":"01a2d0710c7a018c8413cd5dc754e161caadd77b92784d6eea6a60ed7099483e"} Sep 17 13:45:05.472250 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.472174 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerDied","Data":"1e389c66c579121846259a62cc4ad825b8b654172874846648225d7936cd0b79"} Sep 17 13:45:05.550308 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.550262 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused" Sep 17 13:45:05.550308 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.550284 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.41:8816: connect: connection refused" Sep 17 13:45:05.550538 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.550258 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.41:6571: connect: connection refused" Sep 17 13:45:05.640442 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.640413 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused" Sep 17 13:45:05.656701 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.656678 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:45:05.812874 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.812843 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls\") pod \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " Sep 17 13:45:05.813025 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.812903 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls\") pod \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " Sep 17 13:45:05.813025 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.812929 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p24b8\" (UniqueName: \"kubernetes.io/projected/7aa386de-5399-4a6b-a4e7-1f015c4d547e-kube-api-access-p24b8\") pod \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " Sep 17 13:45:05.813025 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.812968 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7aa386de-5399-4a6b-a4e7-1f015c4d547e-registry-tls\") pod \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " Sep 17 13:45:05.813025 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.812997 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7aa386de-5399-4a6b-a4e7-1f015c4d547e-feast-data\") pod \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " Sep 17 13:45:05.813260 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.813025 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-offline-tls\") pod \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\" (UID: \"7aa386de-5399-4a6b-a4e7-1f015c4d547e\") " Sep 17 13:45:05.813378 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.813348 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7aa386de-5399-4a6b-a4e7-1f015c4d547e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7aa386de-5399-4a6b-a4e7-1f015c4d547e" (UID: "7aa386de-5399-4a6b-a4e7-1f015c4d547e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 13:45:05.813489 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.813464 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aa386de-5399-4a6b-a4e7-1f015c4d547e-feast-data" (OuterVolumeSpecName: "feast-data") pod "7aa386de-5399-4a6b-a4e7-1f015c4d547e" (UID: "7aa386de-5399-4a6b-a4e7-1f015c4d547e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:45:05.816080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.816056 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7aa386de-5399-4a6b-a4e7-1f015c4d547e" (UID: "7aa386de-5399-4a6b-a4e7-1f015c4d547e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:45:05.816080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.816035 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls" (OuterVolumeSpecName: "online-tls") pod "7aa386de-5399-4a6b-a4e7-1f015c4d547e" (UID: "7aa386de-5399-4a6b-a4e7-1f015c4d547e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:45:05.816080 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.816076 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7aa386de-5399-4a6b-a4e7-1f015c4d547e" (UID: "7aa386de-5399-4a6b-a4e7-1f015c4d547e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:45:05.816254 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.816090 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aa386de-5399-4a6b-a4e7-1f015c4d547e-kube-api-access-p24b8" (OuterVolumeSpecName: "kube-api-access-p24b8") pod "7aa386de-5399-4a6b-a4e7-1f015c4d547e" (UID: "7aa386de-5399-4a6b-a4e7-1f015c4d547e"). InnerVolumeSpecName "kube-api-access-p24b8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:45:05.839776 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.839754 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:45:05.914418 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.914375 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:05.914418 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.914413 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p24b8\" (UniqueName: \"kubernetes.io/projected/7aa386de-5399-4a6b-a4e7-1f015c4d547e-kube-api-access-p24b8\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:05.914418 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.914422 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7aa386de-5399-4a6b-a4e7-1f015c4d547e-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:05.914418 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.914432 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7aa386de-5399-4a6b-a4e7-1f015c4d547e-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:05.914687 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.914439 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:05.914687 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:05.914447 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7aa386de-5399-4a6b-a4e7-1f015c4d547e-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:06.015303 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.015252 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d65749c2-cc9b-4ae1-aaa1-01b890f16076-feast-data\") pod \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " Sep 17 13:45:06.015460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.015332 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-online-tls\") pod \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " Sep 17 13:45:06.015460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.015361 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-ui-tls\") pod \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " Sep 17 13:45:06.015460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.015392 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-registry-tls\") pod \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " Sep 17 13:45:06.015460 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.015429 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-offline-tls\") pod \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " Sep 17 13:45:06.015651 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.015466 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzklh\" (UniqueName: \"kubernetes.io/projected/d65749c2-cc9b-4ae1-aaa1-01b890f16076-kube-api-access-mzklh\") pod \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\" (UID: \"d65749c2-cc9b-4ae1-aaa1-01b890f16076\") " Sep 17 13:45:06.015829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.015804 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65749c2-cc9b-4ae1-aaa1-01b890f16076-feast-data" (OuterVolumeSpecName: "feast-data") pod "d65749c2-cc9b-4ae1-aaa1-01b890f16076" (UID: "d65749c2-cc9b-4ae1-aaa1-01b890f16076"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:45:06.017735 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.017711 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d65749c2-cc9b-4ae1-aaa1-01b890f16076-kube-api-access-mzklh" (OuterVolumeSpecName: "kube-api-access-mzklh") pod "d65749c2-cc9b-4ae1-aaa1-01b890f16076" (UID: "d65749c2-cc9b-4ae1-aaa1-01b890f16076"). InnerVolumeSpecName "kube-api-access-mzklh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:45:06.017980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.017947 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d65749c2-cc9b-4ae1-aaa1-01b890f16076" (UID: "d65749c2-cc9b-4ae1-aaa1-01b890f16076"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:45:06.018077 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.017965 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d65749c2-cc9b-4ae1-aaa1-01b890f16076" (UID: "d65749c2-cc9b-4ae1-aaa1-01b890f16076"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:45:06.018077 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.018025 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-online-tls" (OuterVolumeSpecName: "online-tls") pod "d65749c2-cc9b-4ae1-aaa1-01b890f16076" (UID: "d65749c2-cc9b-4ae1-aaa1-01b890f16076"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:45:06.018077 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.018037 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d65749c2-cc9b-4ae1-aaa1-01b890f16076" (UID: "d65749c2-cc9b-4ae1-aaa1-01b890f16076"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 13:45:06.116182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.116155 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d65749c2-cc9b-4ae1-aaa1-01b890f16076-feast-data\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:06.116182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.116178 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-online-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:06.116182 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.116186 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-ui-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:06.116410 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.116193 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-registry-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:06.116410 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.116201 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d65749c2-cc9b-4ae1-aaa1-01b890f16076-offline-tls\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:06.116410 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.116208 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzklh\" (UniqueName: \"kubernetes.io/projected/d65749c2-cc9b-4ae1-aaa1-01b890f16076-kube-api-access-mzklh\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:06.478377 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.478288 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" event={"ID":"7aa386de-5399-4a6b-a4e7-1f015c4d547e","Type":"ContainerDied","Data":"cf5671f0bfa1066cd4eb4b382a928ded9a899a717e0a55dd81ae32bab883621a"} Sep 17 13:45:06.478377 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.478347 2575 scope.go:117] "RemoveContainer" containerID="d61b31fe40ca6cda3e609c29d870c5686d4b71d6c5354af3e12623e64c02fb3f" Sep 17 13:45:06.478377 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.478299 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s" Sep 17 13:45:06.481309 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.481267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" event={"ID":"d65749c2-cc9b-4ae1-aaa1-01b890f16076","Type":"ContainerDied","Data":"9395ca43a3ce45e6f297757195b041f75da5edc87687c1a07c135186ad6f2e53"} Sep 17 13:45:06.481409 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.481327 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr" Sep 17 13:45:06.487761 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.487743 2575 scope.go:117] "RemoveContainer" containerID="898085082fc4e50c94e771a31e74e1e2d5b814fcd56baccc7e2382299d47df7d" Sep 17 13:45:06.495522 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.495505 2575 scope.go:117] "RemoveContainer" containerID="fbe597d4f9a72732911586656c824570042f1992b831800f730fc54ee740e013" Sep 17 13:45:06.502700 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.502658 2575 scope.go:117] "RemoveContainer" containerID="d15e68d303433a0ec8af5fedd47eac3c158c5cc1dec64b338b2a3821f2e7a36e" Sep 17 13:45:06.505245 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.505225 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s"] Sep 17 13:45:06.510509 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.510493 2575 scope.go:117] "RemoveContainer" containerID="58ccb675a104136b733f22ff6198623a0b423d5d339490e190d280c3732b2760" Sep 17 13:45:06.515212 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.515193 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-5bb4s"] Sep 17 13:45:06.518479 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.518464 2575 scope.go:117] "RemoveContainer" containerID="c74b49e1282562e3d2a744e36760e5dde2be70bfb2b80c55a5601c06b43f1273" Sep 17 13:45:06.525420 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.525406 2575 scope.go:117] "RemoveContainer" containerID="01a2d0710c7a018c8413cd5dc754e161caadd77b92784d6eea6a60ed7099483e" Sep 17 13:45:06.528975 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.528956 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr"] Sep 17 13:45:06.533648 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.533629 2575 scope.go:117] "RemoveContainer" containerID="dbd8ac38b0f0058ba50f172f4a99540afe819663384bf375041b43b58164e375" Sep 17 13:45:06.534630 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.534613 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ctzhr"] Sep 17 13:45:06.540954 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.540938 2575 scope.go:117] "RemoveContainer" containerID="1e389c66c579121846259a62cc4ad825b8b654172874846648225d7936cd0b79" Sep 17 13:45:06.547892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.547877 2575 scope.go:117] "RemoveContainer" containerID="fc116fe9c85dec12fd372f39645c0af1525219fa2af21790c95a6786b7e34f53" Sep 17 13:45:06.554524 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:06.554509 2575 scope.go:117] "RemoveContainer" containerID="0106d072cb313058d91451b18f0698fdf27bd9b44626655496e9280835b98022" Sep 17 13:45:07.607337 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:07.607306 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" path="/var/lib/kubelet/pods/7aa386de-5399-4a6b-a4e7-1f015c4d547e/volumes" Sep 17 13:45:07.607864 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:07.607848 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" path="/var/lib/kubelet/pods/d65749c2-cc9b-4ae1-aaa1-01b890f16076/volumes" Sep 17 13:45:24.267815 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.267777 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lrrkt/must-gather-vd9t2"] Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268229 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268247 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268266 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="offline" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268294 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="offline" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268303 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="registry" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268312 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="registry" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268322 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268329 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268339 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="feast-apply" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268347 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="feast-apply" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268356 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268364 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268372 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="offline" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268379 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="offline" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268391 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="feast-apply" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268400 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="feast-apply" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268412 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="feast-init" Sep 17 13:45:24.268430 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268419 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="feast-init" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268442 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268450 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268458 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="feast-init" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268465 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="feast-init" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268560 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="ui" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268576 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="online" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268586 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="registry" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268595 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="ui" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268605 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="online" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268613 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aa386de-5399-4a6b-a4e7-1f015c4d547e" containerName="offline" Sep 17 13:45:24.269312 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.268622 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d65749c2-cc9b-4ae1-aaa1-01b890f16076" containerName="offline" Sep 17 13:45:24.271466 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.271446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:24.274007 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.273979 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lrrkt\"/\"default-dockercfg-6kv8x\"" Sep 17 13:45:24.274502 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.274487 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lrrkt\"/\"openshift-service-ca.crt\"" Sep 17 13:45:24.274568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.274509 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lrrkt\"/\"kube-root-ca.crt\"" Sep 17 13:45:24.278574 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.278488 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lrrkt/must-gather-vd9t2"] Sep 17 13:45:24.354142 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.354107 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5d265cf-219a-49f9-8a7c-3ea35614dabc-must-gather-output\") pod \"must-gather-vd9t2\" (UID: \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\") " pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:24.354357 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.354155 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqw7k\" (UniqueName: \"kubernetes.io/projected/c5d265cf-219a-49f9-8a7c-3ea35614dabc-kube-api-access-lqw7k\") pod \"must-gather-vd9t2\" (UID: \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\") " pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:24.455234 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.455200 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5d265cf-219a-49f9-8a7c-3ea35614dabc-must-gather-output\") pod \"must-gather-vd9t2\" (UID: \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\") " pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:24.455419 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.455250 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqw7k\" (UniqueName: \"kubernetes.io/projected/c5d265cf-219a-49f9-8a7c-3ea35614dabc-kube-api-access-lqw7k\") pod \"must-gather-vd9t2\" (UID: \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\") " pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:24.455625 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.455604 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5d265cf-219a-49f9-8a7c-3ea35614dabc-must-gather-output\") pod \"must-gather-vd9t2\" (UID: \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\") " pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:24.463557 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.463528 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqw7k\" (UniqueName: \"kubernetes.io/projected/c5d265cf-219a-49f9-8a7c-3ea35614dabc-kube-api-access-lqw7k\") pod \"must-gather-vd9t2\" (UID: \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\") " pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:24.590772 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.590712 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:24.858751 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.858550 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lrrkt/must-gather-vd9t2"] Sep 17 13:45:24.861723 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:45:24.861696 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5d265cf_219a_49f9_8a7c_3ea35614dabc.slice/crio-a71cdc900e98c328b2c857d9f7489acba8cd46a75fe9cc8795d795d3d52cecbb WatchSource:0}: Error finding container a71cdc900e98c328b2c857d9f7489acba8cd46a75fe9cc8795d795d3d52cecbb: Status 404 returned error can't find the container with id a71cdc900e98c328b2c857d9f7489acba8cd46a75fe9cc8795d795d3d52cecbb Sep 17 13:45:24.863910 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:24.863895 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 13:45:25.562169 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:25.562088 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" event={"ID":"c5d265cf-219a-49f9-8a7c-3ea35614dabc","Type":"ContainerStarted","Data":"a71cdc900e98c328b2c857d9f7489acba8cd46a75fe9cc8795d795d3d52cecbb"} Sep 17 13:45:31.593603 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:31.593550 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" event={"ID":"c5d265cf-219a-49f9-8a7c-3ea35614dabc","Type":"ContainerStarted","Data":"d5750e895c740028103ce3fbb25b65a69d45563b8363b7c4eeb87b7fd71c5094"} Sep 17 13:45:31.593603 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:31.593606 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" event={"ID":"c5d265cf-219a-49f9-8a7c-3ea35614dabc","Type":"ContainerStarted","Data":"2651939b0f254c3706f3e00dc1b71f95ea9769182addf12f5f062248dac6bf8c"} Sep 17 13:45:31.612194 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:31.612141 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" podStartSLOduration=1.887939148 podStartE2EDuration="7.612126007s" totalImagesPullingTime="5.724186859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:45:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:45:31.61031504 +0000 UTC m=+1840.514883717" watchObservedRunningTime="2026-09-17 13:45:31.612126007 +0000 UTC m=+1840.516694685" Sep 17 13:45:40.633923 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:40.633890 2575 generic.go:356] "Generic (PLEG): container finished" podID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerID="2651939b0f254c3706f3e00dc1b71f95ea9769182addf12f5f062248dac6bf8c" exitCode=0 Sep 17 13:45:40.634380 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:40.633953 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" event={"ID":"c5d265cf-219a-49f9-8a7c-3ea35614dabc","Type":"ContainerDied","Data":"2651939b0f254c3706f3e00dc1b71f95ea9769182addf12f5f062248dac6bf8c"} Sep 17 13:45:40.634380 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:40.634253 2575 scope.go:117] "RemoveContainer" containerID="2651939b0f254c3706f3e00dc1b71f95ea9769182addf12f5f062248dac6bf8c" Sep 17 13:45:41.085552 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.085524 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lrrkt_must-gather-vd9t2_c5d265cf-219a-49f9-8a7c-3ea35614dabc/gather/0.log" Sep 17 13:45:41.569477 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.569444 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ww2bp/must-gather-cv6r9"] Sep 17 13:45:41.572767 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.572752 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww2bp/must-gather-cv6r9" Sep 17 13:45:41.574891 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.574872 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ww2bp\"/\"default-dockercfg-pwb9d\"" Sep 17 13:45:41.575014 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.574870 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ww2bp\"/\"kube-root-ca.crt\"" Sep 17 13:45:41.575014 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.574870 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ww2bp\"/\"openshift-service-ca.crt\"" Sep 17 13:45:41.583174 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.583156 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww2bp/must-gather-cv6r9"] Sep 17 13:45:41.709996 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.709966 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd13fd47-2563-4f22-a92b-d13e842aca28-must-gather-output\") pod \"must-gather-cv6r9\" (UID: \"fd13fd47-2563-4f22-a92b-d13e842aca28\") " pod="openshift-must-gather-ww2bp/must-gather-cv6r9" Sep 17 13:45:41.710393 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.710003 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlgdn\" (UniqueName: \"kubernetes.io/projected/fd13fd47-2563-4f22-a92b-d13e842aca28-kube-api-access-rlgdn\") pod \"must-gather-cv6r9\" (UID: \"fd13fd47-2563-4f22-a92b-d13e842aca28\") " pod="openshift-must-gather-ww2bp/must-gather-cv6r9" Sep 17 13:45:41.811136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.811103 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd13fd47-2563-4f22-a92b-d13e842aca28-must-gather-output\") pod \"must-gather-cv6r9\" (UID: \"fd13fd47-2563-4f22-a92b-d13e842aca28\") " pod="openshift-must-gather-ww2bp/must-gather-cv6r9" Sep 17 13:45:41.811136 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.811139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlgdn\" (UniqueName: \"kubernetes.io/projected/fd13fd47-2563-4f22-a92b-d13e842aca28-kube-api-access-rlgdn\") pod \"must-gather-cv6r9\" (UID: \"fd13fd47-2563-4f22-a92b-d13e842aca28\") " pod="openshift-must-gather-ww2bp/must-gather-cv6r9" Sep 17 13:45:41.811508 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.811486 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd13fd47-2563-4f22-a92b-d13e842aca28-must-gather-output\") pod \"must-gather-cv6r9\" (UID: \"fd13fd47-2563-4f22-a92b-d13e842aca28\") " pod="openshift-must-gather-ww2bp/must-gather-cv6r9" Sep 17 13:45:41.820809 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.820764 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlgdn\" (UniqueName: \"kubernetes.io/projected/fd13fd47-2563-4f22-a92b-d13e842aca28-kube-api-access-rlgdn\") pod \"must-gather-cv6r9\" (UID: \"fd13fd47-2563-4f22-a92b-d13e842aca28\") " pod="openshift-must-gather-ww2bp/must-gather-cv6r9" Sep 17 13:45:41.881133 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:41.881111 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww2bp/must-gather-cv6r9" Sep 17 13:45:42.148337 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:42.148243 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww2bp/must-gather-cv6r9"] Sep 17 13:45:42.151867 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:45:42.151835 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd13fd47_2563_4f22_a92b_d13e842aca28.slice/crio-fe49da8c432d4dae94157a408e680940b39f4e77ddd0944bd5edb30849de27d9 WatchSource:0}: Error finding container fe49da8c432d4dae94157a408e680940b39f4e77ddd0944bd5edb30849de27d9: Status 404 returned error can't find the container with id fe49da8c432d4dae94157a408e680940b39f4e77ddd0944bd5edb30849de27d9 Sep 17 13:45:42.642400 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:42.642366 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww2bp/must-gather-cv6r9" event={"ID":"fd13fd47-2563-4f22-a92b-d13e842aca28","Type":"ContainerStarted","Data":"fe49da8c432d4dae94157a408e680940b39f4e77ddd0944bd5edb30849de27d9"} Sep 17 13:45:43.649046 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:43.649011 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww2bp/must-gather-cv6r9" event={"ID":"fd13fd47-2563-4f22-a92b-d13e842aca28","Type":"ContainerStarted","Data":"8454c46d895d064de4f22a15997cbe56b8a918d3724ba53de0d5fc6f11a8b0fe"} Sep 17 13:45:43.649046 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:43.649061 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww2bp/must-gather-cv6r9" event={"ID":"fd13fd47-2563-4f22-a92b-d13e842aca28","Type":"ContainerStarted","Data":"6848f10ff3b546dc795a634f68a16e5ec29d649b4c45c8c640dd495413296a96"} Sep 17 13:45:43.664248 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:43.664199 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ww2bp/must-gather-cv6r9" podStartSLOduration=1.7762915270000001 podStartE2EDuration="2.66418427s" totalImagesPullingTime="887.892743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:45:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:45:43.66319474 +0000 UTC m=+1852.567763427" watchObservedRunningTime="2026-09-17 13:45:43.66418427 +0000 UTC m=+1852.568752946" Sep 17 13:45:44.533495 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:44.533460 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p4p6z_ccee0a47-fcb3-4f7c-a9d5-1cb84a6de084/global-pull-secret-syncer/0.log" Sep 17 13:45:44.616027 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:44.615996 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qvpbl_b1844561-1e55-48e3-aee5-a2af535b8003/konnectivity-agent/0.log" Sep 17 13:45:44.670955 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:44.670926 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-247.ec2.internal_7ad660f1163c376144183c1081f4e9aa/haproxy/0.log" Sep 17 13:45:46.408263 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.408215 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-lrrkt/must-gather-vd9t2"] Sep 17 13:45:46.408849 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.408474 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerName="copy" containerID="cri-o://d5750e895c740028103ce3fbb25b65a69d45563b8363b7c4eeb87b7fd71c5094" gracePeriod=2 Sep 17 13:45:46.410450 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:45:46.410406 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-vd9t2\" is forbidden: User \"system:node:ip-10-0-133-247.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lrrkt\": no relationship found between node 'ip-10-0-133-247.ec2.internal' and this object" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:46.412149 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.412083 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-lrrkt/must-gather-vd9t2"] Sep 17 13:45:46.676639 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.675195 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lrrkt_must-gather-vd9t2_c5d265cf-219a-49f9-8a7c-3ea35614dabc/copy/0.log" Sep 17 13:45:46.676639 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.675662 2575 generic.go:356] "Generic (PLEG): container finished" podID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerID="d5750e895c740028103ce3fbb25b65a69d45563b8363b7c4eeb87b7fd71c5094" exitCode=143 Sep 17 13:45:46.776296 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.773901 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lrrkt_must-gather-vd9t2_c5d265cf-219a-49f9-8a7c-3ea35614dabc/copy/0.log" Sep 17 13:45:46.778045 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.777867 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:46.784618 ip-10-0-133-247 kubenswrapper[2575]: E0917 13:45:46.781020 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-vd9t2\" is forbidden: User \"system:node:ip-10-0-133-247.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lrrkt\": no relationship found between node 'ip-10-0-133-247.ec2.internal' and this object" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:46.866535 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.866494 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqw7k\" (UniqueName: \"kubernetes.io/projected/c5d265cf-219a-49f9-8a7c-3ea35614dabc-kube-api-access-lqw7k\") pod \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\" (UID: \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\") " Sep 17 13:45:46.866733 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.866560 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5d265cf-219a-49f9-8a7c-3ea35614dabc-must-gather-output\") pod \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\" (UID: \"c5d265cf-219a-49f9-8a7c-3ea35614dabc\") " Sep 17 13:45:46.867165 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.867134 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d265cf-219a-49f9-8a7c-3ea35614dabc-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "c5d265cf-219a-49f9-8a7c-3ea35614dabc" (UID: "c5d265cf-219a-49f9-8a7c-3ea35614dabc"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 13:45:46.869932 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.869902 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d265cf-219a-49f9-8a7c-3ea35614dabc-kube-api-access-lqw7k" (OuterVolumeSpecName: "kube-api-access-lqw7k") pod "c5d265cf-219a-49f9-8a7c-3ea35614dabc" (UID: "c5d265cf-219a-49f9-8a7c-3ea35614dabc"). InnerVolumeSpecName "kube-api-access-lqw7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 13:45:46.968101 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.968011 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqw7k\" (UniqueName: \"kubernetes.io/projected/c5d265cf-219a-49f9-8a7c-3ea35614dabc-kube-api-access-lqw7k\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:46.968101 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:46.968053 2575 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c5d265cf-219a-49f9-8a7c-3ea35614dabc-must-gather-output\") on node \"ip-10-0-133-247.ec2.internal\" DevicePath \"\"" Sep 17 13:45:47.611296 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:47.611240 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" path="/var/lib/kubelet/pods/c5d265cf-219a-49f9-8a7c-3ea35614dabc/volumes" Sep 17 13:45:47.682978 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:47.682945 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lrrkt_must-gather-vd9t2_c5d265cf-219a-49f9-8a7c-3ea35614dabc/copy/0.log" Sep 17 13:45:47.683396 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:47.683376 2575 scope.go:117] "RemoveContainer" containerID="d5750e895c740028103ce3fbb25b65a69d45563b8363b7c4eeb87b7fd71c5094" Sep 17 13:45:47.683562 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:47.683546 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lrrkt/must-gather-vd9t2" Sep 17 13:45:47.706497 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:47.705177 2575 scope.go:117] "RemoveContainer" containerID="2651939b0f254c3706f3e00dc1b71f95ea9769182addf12f5f062248dac6bf8c" Sep 17 13:45:48.589297 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.589247 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_256c53e3-30b9-4a95-b585-c7788d5f4fe5/alertmanager/0.log" Sep 17 13:45:48.622487 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.622458 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_256c53e3-30b9-4a95-b585-c7788d5f4fe5/config-reloader/0.log" Sep 17 13:45:48.651210 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.651181 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_256c53e3-30b9-4a95-b585-c7788d5f4fe5/kube-rbac-proxy-web/0.log" Sep 17 13:45:48.682162 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.682129 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_256c53e3-30b9-4a95-b585-c7788d5f4fe5/kube-rbac-proxy/0.log" Sep 17 13:45:48.713907 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.713882 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_256c53e3-30b9-4a95-b585-c7788d5f4fe5/kube-rbac-proxy-metric/0.log" Sep 17 13:45:48.763712 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.763684 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_256c53e3-30b9-4a95-b585-c7788d5f4fe5/prom-label-proxy/0.log" Sep 17 13:45:48.793333 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.793303 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_256c53e3-30b9-4a95-b585-c7788d5f4fe5/init-config-reloader/0.log" Sep 17 13:45:48.862411 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.862292 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-8596r_c1d2c119-7d1d-4520-8e87-b05f083d3a12/kube-state-metrics/0.log" Sep 17 13:45:48.887527 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.887485 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-8596r_c1d2c119-7d1d-4520-8e87-b05f083d3a12/kube-rbac-proxy-main/0.log" Sep 17 13:45:48.917485 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.917458 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-8596r_c1d2c119-7d1d-4520-8e87-b05f083d3a12/kube-rbac-proxy-self/0.log" Sep 17 13:45:48.949979 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.949935 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6565d9b67d-kcl5n_0004302a-59e3-4460-b964-d4b225a7e548/metrics-server/0.log" Sep 17 13:45:48.983251 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:48.983203 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-6n9j8_bc97ce9d-1c8b-46bb-842f-36120db3ade5/monitoring-plugin/0.log" Sep 17 13:45:49.017449 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.017422 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44nw6_0cecb85f-7a0c-477f-8e92-34a77ad52f4e/node-exporter/0.log" Sep 17 13:45:49.043395 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.043348 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44nw6_0cecb85f-7a0c-477f-8e92-34a77ad52f4e/kube-rbac-proxy/0.log" Sep 17 13:45:49.072082 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.072051 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44nw6_0cecb85f-7a0c-477f-8e92-34a77ad52f4e/init-textfile/0.log" Sep 17 13:45:49.313062 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.313030 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-sdswb_d4e0e957-6576-4a42-acac-75de9179551e/kube-rbac-proxy-main/0.log" Sep 17 13:45:49.344603 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.344570 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-sdswb_d4e0e957-6576-4a42-acac-75de9179551e/kube-rbac-proxy-self/0.log" Sep 17 13:45:49.375157 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.375131 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-sdswb_d4e0e957-6576-4a42-acac-75de9179551e/openshift-state-metrics/0.log" Sep 17 13:45:49.652446 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.652348 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-vbhf2_4ba07348-1f3b-4938-8e44-b2984fabce39/prometheus-operator/0.log" Sep 17 13:45:49.682994 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.682955 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-vbhf2_4ba07348-1f3b-4938-8e44-b2984fabce39/kube-rbac-proxy/0.log" Sep 17 13:45:49.771455 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.771424 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-699c65d854-48j7l_f7bd1d70-d8d9-448a-a3b8-f25e23f887f5/telemeter-client/0.log" Sep 17 13:45:49.820301 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.820249 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-699c65d854-48j7l_f7bd1d70-d8d9-448a-a3b8-f25e23f887f5/reload/0.log" Sep 17 13:45:49.858918 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.858890 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-699c65d854-48j7l_f7bd1d70-d8d9-448a-a3b8-f25e23f887f5/kube-rbac-proxy/0.log" Sep 17 13:45:49.902095 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.902063 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d77f96749-ctbf6_89cd6a39-2f1f-40bf-89f0-7aec56692c76/thanos-query/0.log" Sep 17 13:45:49.936563 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.936490 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d77f96749-ctbf6_89cd6a39-2f1f-40bf-89f0-7aec56692c76/kube-rbac-proxy-web/0.log" Sep 17 13:45:49.967404 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:49.967378 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d77f96749-ctbf6_89cd6a39-2f1f-40bf-89f0-7aec56692c76/kube-rbac-proxy/0.log" Sep 17 13:45:50.011757 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:50.011724 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d77f96749-ctbf6_89cd6a39-2f1f-40bf-89f0-7aec56692c76/prom-label-proxy/0.log" Sep 17 13:45:50.060344 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:50.060318 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d77f96749-ctbf6_89cd6a39-2f1f-40bf-89f0-7aec56692c76/kube-rbac-proxy-rules/0.log" Sep 17 13:45:50.109193 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:50.109168 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d77f96749-ctbf6_89cd6a39-2f1f-40bf-89f0-7aec56692c76/kube-rbac-proxy-metrics/0.log" Sep 17 13:45:51.489749 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.489712 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c"] Sep 17 13:45:51.490344 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.490160 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerName="gather" Sep 17 13:45:51.490344 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.490176 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerName="gather" Sep 17 13:45:51.490344 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.490199 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerName="copy" Sep 17 13:45:51.490344 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.490208 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerName="copy" Sep 17 13:45:51.490554 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.490411 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerName="gather" Sep 17 13:45:51.490554 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.490430 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5d265cf-219a-49f9-8a7c-3ea35614dabc" containerName="copy" Sep 17 13:45:51.493954 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.493931 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.501439 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.501413 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c"] Sep 17 13:45:51.618515 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.618479 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74l5j\" (UniqueName: \"kubernetes.io/projected/ca90d59c-0950-4e23-b131-a860cf2c07ff-kube-api-access-74l5j\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.618702 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.618545 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-proc\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.618702 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.618589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-lib-modules\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.618702 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.618623 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-sys\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.618702 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.618698 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-podres\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.719963 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.719924 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-proc\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.720146 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.719979 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-lib-modules\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.720146 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.720008 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-sys\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.720146 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.720124 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-podres\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.720327 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.720171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74l5j\" (UniqueName: \"kubernetes.io/projected/ca90d59c-0950-4e23-b131-a860cf2c07ff-kube-api-access-74l5j\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.720568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.720403 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-proc\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.720568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.720465 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-sys\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.720568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.720512 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-podres\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.720568 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.720525 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca90d59c-0950-4e23-b131-a860cf2c07ff-lib-modules\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.728918 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.728894 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74l5j\" (UniqueName: \"kubernetes.io/projected/ca90d59c-0950-4e23-b131-a860cf2c07ff-kube-api-access-74l5j\") pod \"perf-node-gather-daemonset-srn6c\" (UID: \"ca90d59c-0950-4e23-b131-a860cf2c07ff\") " pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:51.805923 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:51.805884 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:52.007579 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:52.007512 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7677987654-qsrns_2f9d1295-a1cc-4189-9758-25963bcc9c82/console/0.log" Sep 17 13:45:52.090611 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:52.090577 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c"] Sep 17 13:45:52.094430 ip-10-0-133-247 kubenswrapper[2575]: W0917 13:45:52.094396 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podca90d59c_0950_4e23_b131_a860cf2c07ff.slice/crio-2aef78afee998628bcf323ebca528cf4d8256ab7c44b903d4ddd511f7b682ccd WatchSource:0}: Error finding container 2aef78afee998628bcf323ebca528cf4d8256ab7c44b903d4ddd511f7b682ccd: Status 404 returned error can't find the container with id 2aef78afee998628bcf323ebca528cf4d8256ab7c44b903d4ddd511f7b682ccd Sep 17 13:45:52.709533 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:52.709501 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" event={"ID":"ca90d59c-0950-4e23-b131-a860cf2c07ff","Type":"ContainerStarted","Data":"8f497df12396406c6f6e115ddbe3cc4cc1207d7901f8e4d755860276d8bf4a42"} Sep 17 13:45:52.709969 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:52.709551 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" event={"ID":"ca90d59c-0950-4e23-b131-a860cf2c07ff","Type":"ContainerStarted","Data":"2aef78afee998628bcf323ebca528cf4d8256ab7c44b903d4ddd511f7b682ccd"} Sep 17 13:45:52.709969 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:52.709575 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:45:52.725834 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:52.725784 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" podStartSLOduration=1.725766289 podStartE2EDuration="1.725766289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:45:52.72433395 +0000 UTC m=+1861.628902649" watchObservedRunningTime="2026-09-17 13:45:52.725766289 +0000 UTC m=+1861.630334967" Sep 17 13:45:53.387356 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:53.387331 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r9d4n_644b480b-08f1-4fe9-b3db-177aaa8bcf46/dns/0.log" Sep 17 13:45:53.412902 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:53.412881 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r9d4n_644b480b-08f1-4fe9-b3db-177aaa8bcf46/kube-rbac-proxy/0.log" Sep 17 13:45:53.493811 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:53.493784 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qmw76_3319af9c-edac-4f9e-8e6c-9bc9feb432ec/dns-node-resolver/0.log" Sep 17 13:45:53.970458 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:53.970428 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6tpjm_e7d54d52-b7fa-4cc0-a27b-4da60b500985/node-ca/0.log" Sep 17 13:45:55.109689 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:55.109656 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9lbh4_acaee4a3-6596-4bdf-83e3-438894602170/serve-healthcheck-canary/0.log" Sep 17 13:45:55.517915 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:55.517888 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8f5x_111aa84c-189b-489c-b075-226fee25b80d/kube-rbac-proxy/0.log" Sep 17 13:45:55.545829 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:55.545802 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8f5x_111aa84c-189b-489c-b075-226fee25b80d/exporter/0.log" Sep 17 13:45:55.571207 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:55.571184 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8f5x_111aa84c-189b-489c-b075-226fee25b80d/extractor/0.log" Sep 17 13:45:58.725661 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:45:58.725633 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ww2bp/perf-node-gather-daemonset-srn6c" Sep 17 13:46:01.647049 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:01.647022 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8ndwf_97385e5c-6b79-4d4f-9a52-49aace78ffaf/kube-multus/0.log" Sep 17 13:46:01.880775 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:01.880751 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn7zx_9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b/kube-multus-additional-cni-plugins/0.log" Sep 17 13:46:01.909892 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:01.909818 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn7zx_9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b/egress-router-binary-copy/0.log" Sep 17 13:46:01.944980 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:01.944950 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn7zx_9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b/cni-plugins/0.log" Sep 17 13:46:01.977090 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:01.977064 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn7zx_9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b/bond-cni-plugin/0.log" Sep 17 13:46:02.004506 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:02.004483 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn7zx_9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b/routeoverride-cni/0.log" Sep 17 13:46:02.033784 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:02.033741 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn7zx_9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b/whereabouts-cni-bincopy/0.log" Sep 17 13:46:02.064433 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:02.064399 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn7zx_9c1d3d91-1e42-4ef4-8f2c-addd5949ce8b/whereabouts-cni/0.log" Sep 17 13:46:02.387955 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:02.387926 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lnvpk_9e063f7d-b387-4af5-944e-d77ffefceb18/network-metrics-daemon/0.log" Sep 17 13:46:02.417810 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:02.417788 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lnvpk_9e063f7d-b387-4af5-944e-d77ffefceb18/kube-rbac-proxy/0.log" Sep 17 13:46:03.867289 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:03.867255 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-controller/0.log" Sep 17 13:46:03.892559 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:03.892464 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/1.log" Sep 17 13:46:03.892559 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:03.892510 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovn-acl-logging/0.log" Sep 17 13:46:03.910918 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:03.910897 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/kube-rbac-proxy-node/0.log" Sep 17 13:46:03.941059 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:03.941032 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 13:46:03.966584 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:03.966567 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/northd/0.log" Sep 17 13:46:03.995578 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:03.995557 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/nbdb/0.log" Sep 17 13:46:04.021187 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:04.021171 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/sbdb/0.log" Sep 17 13:46:04.066851 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:04.066829 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wh9bs_bcccedee-c77d-4596-a8b4-f304f9c38c13/ovnkube-controller/0.log" Sep 17 13:46:05.071313 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:05.071257 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-2f8lr_cca57ec8-2a52-4896-a70d-fdd82708f7f4/network-check-target-container/0.log" Sep 17 13:46:06.001487 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:06.001458 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-rb8tb_02945a85-1d01-43ae-ab66-57114b5b8a0e/iptables-alerter/0.log" Sep 17 13:46:06.725702 ip-10-0-133-247 kubenswrapper[2575]: I0917 13:46:06.725676 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-wpz88_1b10f405-2e76-4f8c-a846-987bf8f131e8/tuned/0.log"