Sep 15 06:36:34.750850 ip-10-0-133-232 systemd[1]: Starting Kubernetes Kubelet... Sep 15 06:36:35.085054 ip-10-0-133-232 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 06:36:35.085054 ip-10-0-133-232 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 15 06:36:35.085054 ip-10-0-133-232 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 06:36:35.085054 ip-10-0-133-232 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 15 06:36:35.085054 ip-10-0-133-232 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 15 06:36:35.086322 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.086237 2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 15 06:36:35.090109 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090093 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:35.090109 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090108 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090112 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090115 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090118 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090121 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090124 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090127 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090129 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090133 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090135 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090138 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090141 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090143 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090146 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090149 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090152 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090154 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090159 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090163 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:35.090166 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090167 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090172 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090179 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090182 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090185 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090188 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090191 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090194 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090196 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090201 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090204 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090207 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090210 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090213 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090216 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090218 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090221 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090224 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090227 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090230 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:35.090622 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090233 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090236 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090240 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090243 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090246 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090249 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090251 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090254 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090256 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090259 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090262 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090264 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090267 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090269 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090272 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090275 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090278 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090281 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090283 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090286 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:35.091139 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090289 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090292 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090294 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090297 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090300 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090302 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090305 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090307 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090310 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090312 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090315 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090318 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090321 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090323 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090326 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090328 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090331 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090333 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090336 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090339 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:35.091720 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090342 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090344 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090347 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090350 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090352 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090355 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090358 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090361 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090364 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090366 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090369 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090372 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090374 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090377 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090379 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090382 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090384 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090387 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.090389 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091092 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:35.092196 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091099 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091101 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091104 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091107 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091110 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091113 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091116 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091119 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091121 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091124 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091127 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091131 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091134 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091137 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091140 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091142 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091145 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091148 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091151 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091154 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:35.092685 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091157 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091159 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091162 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091165 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091167 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091170 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091172 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091175 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091178 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091180 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091183 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091187 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091190 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091194 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091196 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091199 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091202 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091205 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091207 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:35.093185 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091210 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091212 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091215 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091218 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091221 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091224 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091227 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091229 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091233 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091235 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091238 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091241 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091243 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091246 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091249 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091251 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091254 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091256 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091259 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091261 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:35.093658 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091264 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091266 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091268 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091271 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091273 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091276 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091278 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091281 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091283 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091286 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091288 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091292 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091295 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091298 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091302 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091306 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091309 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091313 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091315 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091318 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:35.094152 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091320 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091323 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091326 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091328 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091331 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091333 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091336 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091338 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091341 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091343 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091346 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091348 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091350 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091353 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091356 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091359 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091361 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091364 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091366 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091748 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:35.094650 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091754 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091757 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091760 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091767 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091770 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091772 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091775 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091778 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091781 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091783 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091786 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091789 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091793 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091797 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091800 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091803 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091806 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091809 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091811 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:35.095151 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091814 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091816 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091819 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091822 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091824 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091827 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091829 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091832 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091836 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091840 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091843 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091845 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091848 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091851 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091854 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091856 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091859 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091862 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091865 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:35.095641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091868 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091870 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091873 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091875 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091878 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091881 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091883 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091886 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091888 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091891 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091893 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091896 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091898 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091901 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091903 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091906 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091908 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091911 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091913 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091916 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:35.096092 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091918 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091921 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091923 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091926 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091928 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091933 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091936 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091940 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091943 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091945 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091948 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091951 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091954 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091956 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091958 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091961 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091963 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091966 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091968 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091971 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:35.096620 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091973 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091976 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091978 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091981 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091984 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091987 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091990 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091993 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091995 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.091998 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092000 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092003 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092005 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092008 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092010 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092012 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092015 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092019 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092021 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092027 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:35.097098 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092115 2552 flags.go:64] FLAG: --address="0.0.0.0" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092158 2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092165 2552 flags.go:64] FLAG: --anonymous-auth="true" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092170 2552 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092175 2552 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092178 2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092182 2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092187 2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092190 2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092193 2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092196 2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092200 2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092203 2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092206 2552 flags.go:64] FLAG: --cgroup-root="" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092209 2552 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092213 2552 flags.go:64] FLAG: --client-ca-file="" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092216 2552 flags.go:64] FLAG: --cloud-provider="external" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092219 2552 flags.go:64] FLAG: --cluster-dns="[]" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092223 2552 flags.go:64] FLAG: --cluster-domain="" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092226 2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092229 2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092232 2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092236 2552 flags.go:64] FLAG: --container-log-max-files="5" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092240 2552 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 15 06:36:35.097593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092243 2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092246 2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092249 2552 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092252 2552 flags.go:64] FLAG: --contention-profiling="false" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092255 2552 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092261 2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092264 2552 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092268 2552 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092272 2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092275 2552 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092278 2552 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092281 2552 flags.go:64] FLAG: --enable-load-reader="false" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092284 2552 flags.go:64] FLAG: --enable-server="true" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092287 2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092292 2552 flags.go:64] FLAG: --event-burst="100" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092295 2552 flags.go:64] FLAG: --event-qps="50" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092298 2552 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092301 2552 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092304 2552 flags.go:64] FLAG: --eviction-hard="" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092308 2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092311 2552 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092314 2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092317 2552 flags.go:64] FLAG: --eviction-soft="" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092320 2552 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092323 2552 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 15 06:36:35.098215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092325 2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092328 2552 flags.go:64] FLAG: --experimental-mounter-path="" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092331 2552 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092334 2552 flags.go:64] FLAG: --fail-swap-on="true" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092337 2552 flags.go:64] FLAG: --feature-gates="" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092340 2552 flags.go:64] FLAG: --file-check-frequency="20s" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092343 2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092346 2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092349 2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092352 2552 flags.go:64] FLAG: --healthz-port="10248" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092355 2552 flags.go:64] FLAG: --help="false" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092358 2552 flags.go:64] FLAG: --hostname-override="ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092363 2552 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092366 2552 flags.go:64] FLAG: --http-check-frequency="20s" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092370 2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092373 2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092377 2552 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092380 2552 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092383 2552 flags.go:64] FLAG: --image-service-endpoint="" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092385 2552 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092388 2552 flags.go:64] FLAG: --kube-api-burst="100" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092392 2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092395 2552 flags.go:64] FLAG: --kube-api-qps="50" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092398 2552 flags.go:64] FLAG: --kube-reserved="" Sep 15 06:36:35.098874 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092401 2552 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092403 2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092406 2552 flags.go:64] FLAG: --kubelet-cgroups="" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092410 2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092412 2552 flags.go:64] FLAG: --lock-file="" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092415 2552 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092418 2552 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092421 2552 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092426 2552 flags.go:64] FLAG: --log-json-split-stream="false" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092429 2552 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092432 2552 flags.go:64] FLAG: --log-text-split-stream="false" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092434 2552 flags.go:64] FLAG: --logging-format="text" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092437 2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092440 2552 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092443 2552 flags.go:64] FLAG: --manifest-url="" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092446 2552 flags.go:64] FLAG: --manifest-url-header="" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092450 2552 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092453 2552 flags.go:64] FLAG: --max-open-files="1000000" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092457 2552 flags.go:64] FLAG: --max-pods="110" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092460 2552 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092464 2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092467 2552 flags.go:64] FLAG: --memory-manager-policy="None" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092471 2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092474 2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092477 2552 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 15 06:36:35.099487 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092480 2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092487 2552 flags.go:64] FLAG: --node-status-max-images="50" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092490 2552 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092493 2552 flags.go:64] FLAG: --oom-score-adj="-999" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092496 2552 flags.go:64] FLAG: --pod-cidr="" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092506 2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092511 2552 flags.go:64] FLAG: --pod-manifest-path="" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092513 2552 flags.go:64] FLAG: --pod-max-pids="-1" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092516 2552 flags.go:64] FLAG: --pods-per-core="0" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092519 2552 flags.go:64] FLAG: --port="10250" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092522 2552 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092526 2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-076725c62e9e59e1a" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092529 2552 flags.go:64] FLAG: --qos-reserved="" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092532 2552 flags.go:64] FLAG: --read-only-port="10255" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092535 2552 flags.go:64] FLAG: --register-node="true" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092538 2552 flags.go:64] FLAG: --register-with-taints="" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092541 2552 flags.go:64] FLAG: --registry-burst="10" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092544 2552 flags.go:64] FLAG: --registry-qps="5" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092547 2552 flags.go:64] FLAG: --reserved-cpus="" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092550 2552 flags.go:64] FLAG: --reserved-memory="" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092554 2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092556 2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092559 2552 flags.go:64] FLAG: --rotate-certificates="false" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092562 2552 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092565 2552 flags.go:64] FLAG: --runonce="false" Sep 15 06:36:35.100114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092568 2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092570 2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092573 2552 flags.go:64] FLAG: --seccomp-default="false" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092577 2552 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092593 2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092596 2552 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092599 2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092603 2552 flags.go:64] FLAG: --storage-driver-password="root" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092605 2552 flags.go:64] FLAG: --storage-driver-secure="false" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092608 2552 flags.go:64] FLAG: --storage-driver-table="stats" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092611 2552 flags.go:64] FLAG: --storage-driver-user="root" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092614 2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092617 2552 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092626 2552 flags.go:64] FLAG: --system-cgroups="" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092629 2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092634 2552 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092637 2552 flags.go:64] FLAG: --tls-cert-file="" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092640 2552 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092646 2552 flags.go:64] FLAG: --tls-min-version="" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092649 2552 flags.go:64] FLAG: --tls-private-key-file="" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092652 2552 flags.go:64] FLAG: --topology-manager-policy="none" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092655 2552 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092658 2552 flags.go:64] FLAG: --topology-manager-scope="container" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092661 2552 flags.go:64] FLAG: --v="2" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092665 2552 flags.go:64] FLAG: --version="false" Sep 15 06:36:35.100743 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092669 2552 flags.go:64] FLAG: --vmodule="" Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092673 2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.092676 2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092811 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092815 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092819 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092822 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092825 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092827 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092830 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092834 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092838 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092841 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092844 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092846 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092849 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092852 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092854 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092857 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092860 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:35.101339 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092862 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092865 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092868 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092871 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092873 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092876 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092878 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092881 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092884 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092886 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092888 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092891 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092894 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092896 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092899 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092901 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092904 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092907 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092909 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092912 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:35.101875 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092914 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092917 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092921 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092925 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092927 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092930 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092932 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092935 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092938 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092941 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092943 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092946 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092948 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092951 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092959 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092962 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092965 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092969 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092973 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092977 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:35.102368 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092981 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.092998 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093001 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093004 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093007 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093011 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093015 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093018 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093020 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093023 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093026 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093029 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093032 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093034 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093038 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093042 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093044 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093047 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093049 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:35.102862 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093053 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093057 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093061 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093064 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093067 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093070 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093072 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093075 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093084 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093087 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093089 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093092 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093095 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093097 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093100 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093102 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093105 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093107 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093110 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:35.103752 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093113 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093115 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093118 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.093120 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.093833 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.100741 2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.100757 2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.100776 2552 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.100782 2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100834 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100840 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100844 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100848 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100850 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100854 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:35.104244 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100857 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100860 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100862 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100865 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100868 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100870 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100873 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100876 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100878 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100881 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100884 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100886 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100889 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100891 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100894 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100897 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100900 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100902 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100905 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100908 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:35.104641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100910 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100913 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100916 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100920 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100922 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100925 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100928 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100931 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100933 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100936 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100938 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100941 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100943 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100946 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100948 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100951 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100953 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100956 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100958 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100961 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:35.105120 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100963 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100966 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100968 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100971 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100974 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100977 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100979 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100982 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100984 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100987 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100989 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100992 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100995 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.100998 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101000 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101003 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101006 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101009 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101012 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101014 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:35.105641 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101017 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101020 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101022 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101025 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101029 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101033 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101037 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101040 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101042 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101045 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101048 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101051 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101054 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101056 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101059 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101062 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101064 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101066 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:35.106149 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101069 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101072 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101074 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101077 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101079 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101082 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101084 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101087 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101090 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101092 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101095 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101098 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101101 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101103 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101106 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:35.106579 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.101111 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101226 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101230 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101233 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101236 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101239 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101242 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101245 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101248 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101251 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101255 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101258 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101261 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101264 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101267 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101270 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101273 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101276 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101279 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101282 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 15 06:36:35.106972 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101284 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101287 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101289 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101292 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101295 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101298 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101300 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101303 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101307 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101311 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101314 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101317 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101319 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101322 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101324 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101327 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101329 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101332 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101335 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101337 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 15 06:36:35.107453 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101340 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101343 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101345 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101349 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101351 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101354 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101357 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101359 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101362 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101364 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101367 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101370 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101373 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101375 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101377 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101380 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101382 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101385 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101388 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101390 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 15 06:36:35.107942 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101393 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101395 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101398 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101401 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101403 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101406 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101408 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101411 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101414 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101418 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101421 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101423 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101426 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101429 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101431 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101434 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101436 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101439 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101441 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 15 06:36:35.108412 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101444 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101446 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101449 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101451 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101454 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101456 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101460 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101462 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101465 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101467 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101470 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101472 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101475 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101477 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101480 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101483 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101486 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101488 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101491 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 15 06:36:35.108933 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101493 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 15 06:36:35.109377 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:35.101496 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 15 06:36:35.109377 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.101500 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 15 06:36:35.109377 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.101613 2552 server.go:962] "Client rotation is on, will bootstrap in background" Sep 15 06:36:35.109377 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.106421 2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 15 06:36:35.109377 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.107395 2552 server.go:1019] "Starting client certificate rotation" Sep 15 06:36:35.109377 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.107488 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 06:36:35.109377 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.107516 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 15 06:36:35.109617 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.109523 2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 06:36:35.110698 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.110681 2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 15 06:36:35.120094 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.120075 2552 log.go:25] "Validated CRI v1 runtime API" Sep 15 06:36:35.125449 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.125431 2552 log.go:25] "Validated CRI v1 image API" Sep 15 06:36:35.126282 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.126263 2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 15 06:36:35.128435 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.128416 2552 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 dc23d83b-59e1-477d-8e93-eee3679ba740:/dev/nvme0n1p4 f9dcabcf-da64-45fe-b2e3-fb34d7f7010a:/dev/nvme0n1p3] Sep 15 06:36:35.128513 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.128433 2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 15 06:36:35.133287 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.133092 2552 manager.go:216] Machine: {Timestamp:2026-09-15 06:36:35.131940983 +0000 UTC m=+0.283980266 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100425 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27c287a13b5562c59db3a0bcbbe6de SystemUUID:ec27c287-a13b-5562-c59d-b3a0bcbbe6de BootID:7fdd874b-7e82-4390-9480-710dba833352 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d5:02:b8:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d5:02:b8:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:6c:8a:ba:e0:fa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 15 06:36:35.133287 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.133284 2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 15 06:36:35.133405 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.133353 2552 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 15 06:36:35.136284 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.136257 2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 15 06:36:35.136425 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.136288 2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-232.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 15 06:36:35.136922 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.136906 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 06:36:35.137331 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.137319 2552 topology_manager.go:138] "Creating topology manager with none policy" Sep 15 06:36:35.137370 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.137334 2552 container_manager_linux.go:309] "Creating device plugin manager" Sep 15 06:36:35.137370 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.137341 2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 06:36:35.138195 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.138166 2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 15 06:36:35.139083 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.139068 2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 15 06:36:35.141887 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.141875 2552 state_mem.go:36] "Initialized new in-memory state store" Sep 15 06:36:35.142016 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.142006 2552 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 15 06:36:35.142053 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.142037 2552 kubelet.go:486] "Attempting to sync node with API server" Sep 15 06:36:35.142053 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.142049 2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 15 06:36:35.142104 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.142060 2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 15 06:36:35.142104 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.142069 2552 kubelet.go:398] "Adding apiserver pod source" Sep 15 06:36:35.142104 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.142077 2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 15 06:36:35.142413 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.142401 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 06:36:35.142472 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.142420 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 15 06:36:35.144272 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.144253 2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 15 06:36:35.146332 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.146318 2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 15 06:36:35.146385 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.146349 2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 15 06:36:35.147789 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147774 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 15 06:36:35.147840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147793 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 15 06:36:35.147840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147799 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 15 06:36:35.147840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147804 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 15 06:36:35.147840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147813 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 15 06:36:35.147840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147818 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 15 06:36:35.147840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147830 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 15 06:36:35.147840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147835 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 15 06:36:35.147840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147842 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 15 06:36:35.148085 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147847 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 15 06:36:35.148085 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147856 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 15 06:36:35.148085 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.147870 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 15 06:36:35.149146 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.149134 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 15 06:36:35.149146 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.149146 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 15 06:36:35.149862 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.149833 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 15 06:36:35.149909 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.149853 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-232.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 15 06:36:35.151446 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.151425 2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-232.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 15 06:36:35.152770 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.152756 2552 server.go:1268] "Started kubelet" Sep 15 06:36:35.152897 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.152834 2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 15 06:36:35.153006 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.152910 2552 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 15 06:36:35.153437 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.152889 2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 15 06:36:35.153771 ip-10-0-133-232 systemd[1]: Started Kubernetes Kubelet. Sep 15 06:36:35.153994 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.153977 2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 15 06:36:35.154550 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.154534 2552 server.go:310] "Adding debug handlers to kubelet server" Sep 15 06:36:35.156481 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.156080 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q4jjx" Sep 15 06:36:35.157599 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.157554 2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 15 06:36:35.160068 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.160053 2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 15 06:36:35.160163 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.160082 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 15 06:36:35.160163 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.160149 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.160623 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.160574 2552 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 15 06:36:35.160708 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.160629 2552 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 15 06:36:35.160708 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.160675 2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 15 06:36:35.160816 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.160714 2552 reconstruct.go:97] "Volume reconstruction finished" Sep 15 06:36:35.160816 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.160722 2552 reconciler.go:29] "Reconciler: start to sync state" Sep 15 06:36:35.161622 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.161607 2552 factory.go:55] Registering systemd factory Sep 15 06:36:35.161683 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.161626 2552 factory.go:223] Registration of the systemd container factory successfully Sep 15 06:36:35.161835 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.161823 2552 factory.go:153] Registering CRI-O factory Sep 15 06:36:35.161835 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.161836 2552 factory.go:223] Registration of the crio container factory successfully Sep 15 06:36:35.161907 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.161882 2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 15 06:36:35.162308 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.162295 2552 factory.go:103] Registering Raw factory Sep 15 06:36:35.162379 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.162314 2552 manager.go:1196] Started watching for new ooms in manager Sep 15 06:36:35.162691 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.162681 2552 manager.go:349] Starting recovery of all containers Sep 15 06:36:35.164342 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.164319 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q4jjx" Sep 15 06:36:35.165754 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.165729 2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-232.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 15 06:36:35.170570 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.170535 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 15 06:36:35.174226 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.170554 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-232.ec2.internal.18d56b1c0be9342d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-232.ec2.internal,UID:ip-10-0-133-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-232.ec2.internal,},FirstTimestamp:2026-09-15 06:36:35.152729133 +0000 UTC m=+0.304768416,LastTimestamp:2026-09-15 06:36:35.152729133 +0000 UTC m=+0.304768416,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-232.ec2.internal,}" Sep 15 06:36:35.175454 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.175436 2552 manager.go:354] Recovery completed Sep 15 06:36:35.177640 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.176657 2552 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 15 06:36:35.179774 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.179760 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:35.181614 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.181596 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:35.181699 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.181634 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:35.181699 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.181653 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:35.182223 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.182207 2552 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 15 06:36:35.182223 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.182221 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:35.182320 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.182222 2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 15 06:36:35.182320 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.182267 2552 state_mem.go:36] "Initialized new in-memory state store" Sep 15 06:36:35.182983 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.182968 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:35.183040 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.182997 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:35.183040 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.183008 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:35.183473 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.183461 2552 policy_none.go:49] "None policy: Start" Sep 15 06:36:35.183505 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.183477 2552 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 15 06:36:35.183505 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.183487 2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 15 06:36:35.184520 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.184510 2552 policy_none.go:47] "Start" Sep 15 06:36:35.215233 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.215215 2552 manager.go:337] "Starting Device Plugin manager" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.215261 2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.215277 2552 server.go:87] "Starting device plugin registration server" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.215634 2552 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.215645 2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.215742 2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.215816 2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.215824 2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.216291 2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 15 06:36:35.232032 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.216340 2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.256718 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.256686 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 15 06:36:35.258005 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.257984 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 15 06:36:35.258123 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.258043 2552 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 15 06:36:35.258123 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.258068 2552 kubelet.go:2448] "Starting kubelet main sync loop" Sep 15 06:36:35.258123 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.258112 2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 15 06:36:35.259907 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.259890 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:35.315750 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.315718 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:35.317198 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.317179 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:35.317325 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.317210 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:35.317325 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.317224 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:35.317325 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.317246 2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.324296 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.324280 2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.324384 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.324301 2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-232.ec2.internal\": node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.333578 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.333557 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.358595 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.358515 2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal"] Sep 15 06:36:35.358668 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.358597 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:35.359918 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.359903 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:35.360000 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.359929 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:35.360000 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.359943 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:35.361177 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361164 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:35.361300 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361272 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.361344 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361317 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:35.361902 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361879 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:35.361989 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361905 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:35.361989 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361918 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:35.361989 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361883 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:35.361989 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361952 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:35.361989 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.361968 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:35.363009 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.362994 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.363086 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.363020 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 15 06:36:35.363949 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.363932 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientMemory" Sep 15 06:36:35.364026 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.363966 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasNoDiskPressure" Sep 15 06:36:35.364026 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.363982 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeHasSufficientPID" Sep 15 06:36:35.396023 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.396005 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-232.ec2.internal\" not found" node="ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.400106 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.400091 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-232.ec2.internal\" not found" node="ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.433949 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.433913 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.461616 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.461580 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bce2d90bf9ede72f5f9236ac85ff5a6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal\" (UID: \"bce2d90bf9ede72f5f9236ac85ff5a6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.461754 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.461628 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bce2d90bf9ede72f5f9236ac85ff5a6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal\" (UID: \"bce2d90bf9ede72f5f9236ac85ff5a6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.534158 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.534130 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.562461 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.562438 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bce2d90bf9ede72f5f9236ac85ff5a6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal\" (UID: \"bce2d90bf9ede72f5f9236ac85ff5a6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.562530 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.562466 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bce2d90bf9ede72f5f9236ac85ff5a6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal\" (UID: \"bce2d90bf9ede72f5f9236ac85ff5a6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.562530 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.562493 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed80a124298f3affca688f00871d8528-config\") pod \"kube-apiserver-proxy-ip-10-0-133-232.ec2.internal\" (UID: \"ed80a124298f3affca688f00871d8528\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.562641 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.562532 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bce2d90bf9ede72f5f9236ac85ff5a6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal\" (UID: \"bce2d90bf9ede72f5f9236ac85ff5a6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.562641 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.562544 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bce2d90bf9ede72f5f9236ac85ff5a6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal\" (UID: \"bce2d90bf9ede72f5f9236ac85ff5a6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.634671 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.634617 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.663146 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.663109 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed80a124298f3affca688f00871d8528-config\") pod \"kube-apiserver-proxy-ip-10-0-133-232.ec2.internal\" (UID: \"ed80a124298f3affca688f00871d8528\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.663226 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.663165 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed80a124298f3affca688f00871d8528-config\") pod \"kube-apiserver-proxy-ip-10-0-133-232.ec2.internal\" (UID: \"ed80a124298f3affca688f00871d8528\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.698256 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.698232 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.703052 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:35.703037 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" Sep 15 06:36:35.735548 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.735522 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.836078 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.836048 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:35.936568 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:35.936502 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:36.037055 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:36.037029 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:36.107538 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:36.107495 2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 15 06:36:36.107987 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:36.107665 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 15 06:36:36.137796 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:36.137772 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:37.767132 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:37.767097 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 06:31:36 +0000 UTC" deadline="2028-06-29 04:48:11.188275294 +0000 UTC" Sep 15 06:36:37.767132 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:37.767128 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15670h11m33.42115032s" Sep 15 06:36:37.835659 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:37.835633 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:37.895115 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:37.895086 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded80a124298f3affca688f00871d8528.slice/crio-7d16efce24000c2d45d13d3b6e3fd1741da84fdce28bd5b5b8108268d202587c WatchSource:0}: Error finding container 7d16efce24000c2d45d13d3b6e3fd1741da84fdce28bd5b5b8108268d202587c: Status 404 returned error can't find the container with id 7d16efce24000c2d45d13d3b6e3fd1741da84fdce28bd5b5b8108268d202587c Sep 15 06:36:37.899074 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:37.898921 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 06:36:37.936230 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:37.936205 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:37.959920 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:37.959897 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:38.009950 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.009925 2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:38.037064 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.037039 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:38.137571 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.137513 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:38.140027 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:38.140005 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce2d90bf9ede72f5f9236ac85ff5a6d.slice/crio-1078346ab42b4185c2eb4c8b28545cf3ee92e7754bea75be42e1280aeadba097 WatchSource:0}: Error finding container 1078346ab42b4185c2eb4c8b28545cf3ee92e7754bea75be42e1280aeadba097: Status 404 returned error can't find the container with id 1078346ab42b4185c2eb4c8b28545cf3ee92e7754bea75be42e1280aeadba097 Sep 15 06:36:38.238033 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.237999 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-232.ec2.internal\" not found" Sep 15 06:36:38.291554 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.291524 2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:38.357781 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.357754 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" Sep 15 06:36:38.367704 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.367686 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 06:36:38.368550 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.368538 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" Sep 15 06:36:38.377188 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.377168 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 15 06:36:38.740671 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.740634 2552 apiserver.go:52] "Watching apiserver" Sep 15 06:36:38.748479 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.748449 2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 15 06:36:38.749514 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.749487 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-kqt7l","openshift-dns/node-resolver-m48nb","openshift-image-registry/node-ca-pjt59","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal","openshift-multus/network-metrics-daemon-ns6cw","kube-system/konnectivity-agent-vtltd","openshift-multus/multus-additional-cni-plugins-6hddd","openshift-multus/multus-qxgjj","openshift-network-diagnostics/network-check-target-pnhgk","openshift-network-operator/iptables-alerter-z87dk","openshift-ovn-kubernetes/ovnkube-node-cgvr8","kube-system/global-pull-secret-syncer-96qh7","kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w"] Sep 15 06:36:38.751210 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.751188 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.752602 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.752582 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.753838 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.753814 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c5z8w\"" Sep 15 06:36:38.754456 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.754047 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 15 06:36:38.754456 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.754123 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 15 06:36:38.754456 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.754134 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.754456 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.754332 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 15 06:36:38.755488 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.755469 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 15 06:36:38.755995 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.755696 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 15 06:36:38.755995 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.755856 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 15 06:36:38.755995 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.755989 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.757551 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.757274 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:38.757551 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.757354 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:38.759058 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.758861 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 15 06:36:38.759058 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.759051 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:38.760452 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.760039 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:38.760452 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.760103 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:38.761961 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.761610 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.775560 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.775536 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8ln8x\"" Sep 15 06:36:38.775734 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.775719 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 15 06:36:38.775918 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.775905 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776109 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776214 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kq78\" (UniqueName: \"kubernetes.io/projected/178866a3-7693-4a4b-8a07-c489b6fed8a7-kube-api-access-6kq78\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776253 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e07529d-1e9e-4024-8870-0400eb65c215-ovn-node-metrics-cert\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776281 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-cnibin\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776320 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776311 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ae71f33-68fa-4077-8b65-38ff766cddc1-konnectivity-ca\") pod \"konnectivity-agent-vtltd\" (UID: \"8ae71f33-68fa-4077-8b65-38ff766cddc1\") " pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776400 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-node-log\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776457 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-log-socket\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776489 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776524 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-system-cni-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776555 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-cni-multus\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776583 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776591 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776615 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776645 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-var-lib-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776670 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e48e916e-5d40-41bc-8359-bc216fd5b2dd-cni-binary-copy\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776699 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-socket-dir-parent\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.778042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776735 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-daemon-config\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776766 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-system-cni-dir\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776797 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkhj6\" (UniqueName: \"kubernetes.io/projected/532a12f1-78ac-4e4d-a02d-6be6c808825f-kube-api-access-rkhj6\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776830 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ae71f33-68fa-4077-8b65-38ff766cddc1-agent-certs\") pod \"konnectivity-agent-vtltd\" (UID: \"8ae71f33-68fa-4077-8b65-38ff766cddc1\") " pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776848 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cmgxq\"" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776858 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f694d292-e504-410d-924d-170bfeb5971c-iptables-alerter-script\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776887 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-kubelet\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776915 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-systemd\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776944 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-ovnkube-script-lib\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.776986 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777021 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f694d292-e504-410d-924d-170bfeb5971c-host-slash\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777052 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-systemd-units\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777079 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-slash\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777104 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777112 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-ovn\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777138 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-cni-bin\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777168 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779061 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777206 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-env-overrides\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777238 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxsvp\" (UniqueName: \"kubernetes.io/projected/f694d292-e504-410d-924d-170bfeb5971c-kube-api-access-lxsvp\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777273 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-etc-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777299 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-ovnkube-config\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777330 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfm5q\" (UniqueName: \"kubernetes.io/projected/4e07529d-1e9e-4024-8870-0400eb65c215-kube-api-access-wfm5q\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777366 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-cni-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777460 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777494 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-cni-bin\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.777536 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777534 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-hostroot\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777579 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-conf-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777612 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-run-netns\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777618 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777638 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777670 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zsm4\" (UniqueName: \"kubernetes.io/projected/e48e916e-5d40-41bc-8359-bc216fd5b2dd-kube-api-access-9zsm4\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777702 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-cnibin\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777743 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777360 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 15 06:36:38.779768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777784 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777831 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-cni-netd\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777863 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-k8s-cni-cncf-io\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.777890 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-netns\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778080 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-kubelet\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778118 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-multus-certs\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778151 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-etc-kubernetes\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778280 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zprl8\"" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778329 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778655 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778692 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778717 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9mtxn\"" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.778908 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.779006 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 15 06:36:38.780747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.779597 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-26rps\"" Sep 15 06:36:38.781464 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.780960 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 15 06:36:38.781464 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.781000 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 15 06:36:38.781464 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.781275 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tzjl6\"" Sep 15 06:36:38.781858 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.781837 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 15 06:36:38.783221 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.783204 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.785547 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.785262 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.785689 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.785672 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 15 06:36:38.785818 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.785804 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 15 06:36:38.785938 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.785922 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jstt7\"" Sep 15 06:36:38.786008 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.785946 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 15 06:36:38.787945 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.787753 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 15 06:36:38.788985 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.788444 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 15 06:36:38.788985 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.788496 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cch5p\"" Sep 15 06:36:38.788985 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.788700 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 15 06:36:38.788985 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.788735 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 15 06:36:38.804746 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.804704 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fb5g7" Sep 15 06:36:38.814501 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.814483 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fb5g7" Sep 15 06:36:38.859392 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.859327 2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 15 06:36:38.865542 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.865481 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" event={"ID":"bce2d90bf9ede72f5f9236ac85ff5a6d","Type":"ContainerStarted","Data":"1078346ab42b4185c2eb4c8b28545cf3ee92e7754bea75be42e1280aeadba097"} Sep 15 06:36:38.867239 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.867213 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" event={"ID":"ed80a124298f3affca688f00871d8528","Type":"ContainerStarted","Data":"7d16efce24000c2d45d13d3b6e3fd1741da84fdce28bd5b5b8108268d202587c"} Sep 15 06:36:38.878803 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878783 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.878901 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878811 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-system-cni-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.878901 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878827 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-cni-multus\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.878901 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878852 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9b78\" (UniqueName: \"kubernetes.io/projected/27055683-09d8-4071-b297-5feb0eb4e3b8-kube-api-access-j9b78\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.878901 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878873 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0401d78e-6529-4987-90cb-4369375d67be-dbus\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.878901 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878886 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e48e916e-5d40-41bc-8359-bc216fd5b2dd-cni-binary-copy\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.878901 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878901 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-daemon-config\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878901 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878915 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878930 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0401d78e-6529-4987-90cb-4369375d67be-kubelet-config\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.878975 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-kubelet\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879008 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-ovnkube-script-lib\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879034 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysctl-d\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879058 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-cni-bin\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879080 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-cni-bin\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879103 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-etc-kubernetes\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879130 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kq78\" (UniqueName: \"kubernetes.io/projected/178866a3-7693-4a4b-8a07-c489b6fed8a7-kube-api-access-6kq78\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879154 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkhj6\" (UniqueName: \"kubernetes.io/projected/532a12f1-78ac-4e4d-a02d-6be6c808825f-kube-api-access-rkhj6\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879178 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-host\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879200 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl68h\" (UniqueName: \"kubernetes.io/projected/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-kube-api-access-kl68h\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879235 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-ovnkube-config\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879259 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfm5q\" (UniqueName: \"kubernetes.io/projected/4e07529d-1e9e-4024-8870-0400eb65c215-kube-api-access-wfm5q\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879281 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-hostroot\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.879309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879305 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e87755-014a-4616-b966-b66c76c86721-host\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.880135 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879344 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tvp2\" (UniqueName: \"kubernetes.io/projected/a0e87755-014a-4616-b966-b66c76c86721-kube-api-access-2tvp2\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.880135 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879372 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-run-netns\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.880135 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879397 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.880135 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879423 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e48e916e-5d40-41bc-8359-bc216fd5b2dd-cni-binary-copy\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.880135 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879550 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-system-cni-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.880135 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879600 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-cni-multus\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.880135 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879662 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-kubelet\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.880135 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879926 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.880512 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.879422 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.880567 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.880516 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-cni-bin\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.880645 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.880562 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-cni-bin\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.880645 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.880600 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-etc-kubernetes\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.880808 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.880788 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-daemon-config\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.880945 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.880913 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s6st\" (UniqueName: \"kubernetes.io/projected/c4e5225b-1c04-42e3-93da-e3bef0b45c73-kube-api-access-5s6st\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.881027 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.880946 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.881027 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881001 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-systemd\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.881125 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881029 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-lib-modules\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.881125 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881054 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-cni-netd\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.881125 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881077 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-netns\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.881125 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881113 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-kubelet\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.881314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881149 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysctl-conf\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.881314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881174 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-cnibin\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.881314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881199 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ae71f33-68fa-4077-8b65-38ff766cddc1-konnectivity-ca\") pod \"konnectivity-agent-vtltd\" (UID: \"8ae71f33-68fa-4077-8b65-38ff766cddc1\") " pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:38.881314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881222 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0e87755-014a-4616-b966-b66c76c86721-serviceca\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.881314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881244 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27055683-09d8-4071-b297-5feb0eb4e3b8-tmp-dir\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.881314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881265 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-sys\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.881314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881287 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-node-log\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.881314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881311 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-log-socket\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881336 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881359 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881400 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-ovnkube-config\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881453 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysconfig\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881481 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-var-lib-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881509 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-socket-dir-parent\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881535 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-system-cni-dir\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881562 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881571 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-ovnkube-script-lib\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881586 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f694d292-e504-410d-924d-170bfeb5971c-iptables-alerter-script\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.881694 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.881689 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:38.882249 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.881748 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:36:39.381728992 +0000 UTC m=+2.936386410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:38.882249 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881922 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-var-lib-kubelet\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.882249 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.881979 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-cni-netd\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.882249 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882013 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-netns\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.882249 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882047 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f694d292-e504-410d-924d-170bfeb5971c-iptables-alerter-script\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.882249 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882054 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-cnibin\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.882249 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882096 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-log-socket\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.882586 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882263 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-hostroot\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.882586 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882538 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ae71f33-68fa-4077-8b65-38ff766cddc1-konnectivity-ca\") pod \"konnectivity-agent-vtltd\" (UID: \"8ae71f33-68fa-4077-8b65-38ff766cddc1\") " pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:38.882684 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882586 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-node-log\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.882684 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882616 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-systemd\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.882684 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882620 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.882684 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882645 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.882684 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882658 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-run-netns\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.882921 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882706 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-systemd\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.882921 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882797 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.882921 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882826 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-socket-dir-parent\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.882921 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882845 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-var-lib-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.882921 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882853 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-system-cni-dir\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.882921 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882887 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-registration-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882921 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-device-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.882990 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f694d292-e504-410d-924d-170bfeb5971c-host-slash\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883023 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-systemd-units\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883046 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-slash\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883086 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-slash\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883095 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-systemd-units\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883114 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-ovn\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883151 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f694d292-e504-410d-924d-170bfeb5971c-host-slash\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883154 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-ovn\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883184 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883228 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883217 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-env-overrides\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883243 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ae71f33-68fa-4077-8b65-38ff766cddc1-agent-certs\") pod \"konnectivity-agent-vtltd\" (UID: \"8ae71f33-68fa-4077-8b65-38ff766cddc1\") " pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883276 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/178866a3-7693-4a4b-8a07-c489b6fed8a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883300 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxsvp\" (UniqueName: \"kubernetes.io/projected/f694d292-e504-410d-924d-170bfeb5971c-kube-api-access-lxsvp\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883339 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-etc-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883379 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-cni-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883404 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-conf-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883429 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zsm4\" (UniqueName: \"kubernetes.io/projected/e48e916e-5d40-41bc-8359-bc216fd5b2dd-kube-api-access-9zsm4\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883450 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-etc-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883480 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.883779 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883591 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e07529d-1e9e-4024-8870-0400eb65c215-env-overrides\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883801 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-socket-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883825 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-cni-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883829 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-sys-fs\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883874 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-run\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883919 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883946 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-cnibin\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.883998 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-var-lib-kubelet\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884022 2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884049 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/178866a3-7693-4a4b-8a07-c489b6fed8a7-cnibin\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884048 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-k8s-cni-cncf-io\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884092 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-multus-conf-dir\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884097 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-k8s-cni-cncf-io\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884125 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e07529d-1e9e-4024-8870-0400eb65c215-run-openvswitch\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884130 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-multus-certs\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884163 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-etc-selinux\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884166 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e48e916e-5d40-41bc-8359-bc216fd5b2dd-host-run-multus-certs\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.884302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884196 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-modprobe-d\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.885144 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884230 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-tuned\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.885144 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884278 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-tmp\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.885144 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884309 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e07529d-1e9e-4024-8870-0400eb65c215-ovn-node-metrics-cert\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.885144 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884345 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27055683-09d8-4071-b297-5feb0eb4e3b8-hosts-file\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.885144 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.884447 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-kubernetes\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.888354 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.888302 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e07529d-1e9e-4024-8870-0400eb65c215-ovn-node-metrics-cert\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.888482 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.888445 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ae71f33-68fa-4077-8b65-38ff766cddc1-agent-certs\") pod \"konnectivity-agent-vtltd\" (UID: \"8ae71f33-68fa-4077-8b65-38ff766cddc1\") " pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:38.891009 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.890940 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkhj6\" (UniqueName: \"kubernetes.io/projected/532a12f1-78ac-4e4d-a02d-6be6c808825f-kube-api-access-rkhj6\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:38.891144 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.891122 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kq78\" (UniqueName: \"kubernetes.io/projected/178866a3-7693-4a4b-8a07-c489b6fed8a7-kube-api-access-6kq78\") pod \"multus-additional-cni-plugins-6hddd\" (UID: \"178866a3-7693-4a4b-8a07-c489b6fed8a7\") " pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:38.893414 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.893373 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfm5q\" (UniqueName: \"kubernetes.io/projected/4e07529d-1e9e-4024-8870-0400eb65c215-kube-api-access-wfm5q\") pod \"ovnkube-node-cgvr8\" (UID: \"4e07529d-1e9e-4024-8870-0400eb65c215\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:38.893727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.893705 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxsvp\" (UniqueName: \"kubernetes.io/projected/f694d292-e504-410d-924d-170bfeb5971c-kube-api-access-lxsvp\") pod \"iptables-alerter-z87dk\" (UID: \"f694d292-e504-410d-924d-170bfeb5971c\") " pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:38.894901 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.894816 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zsm4\" (UniqueName: \"kubernetes.io/projected/e48e916e-5d40-41bc-8359-bc216fd5b2dd-kube-api-access-9zsm4\") pod \"multus-qxgjj\" (UID: \"e48e916e-5d40-41bc-8359-bc216fd5b2dd\") " pod="openshift-multus/multus-qxgjj" Sep 15 06:36:38.984936 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.984900 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-socket-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.984944 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-sys-fs\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.984980 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-run\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985001 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-var-lib-kubelet\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985018 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-etc-selinux\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985033 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-modprobe-d\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985054 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-tuned\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985087 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-tmp\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985107 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27055683-09d8-4071-b297-5feb0eb4e3b8-hosts-file\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985124 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-kubernetes\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985146 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9b78\" (UniqueName: \"kubernetes.io/projected/27055683-09d8-4071-b297-5feb0eb4e3b8-kube-api-access-j9b78\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985161 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0401d78e-6529-4987-90cb-4369375d67be-dbus\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.985169 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985176 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985193 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0401d78e-6529-4987-90cb-4369375d67be-kubelet-config\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985212 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysctl-d\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985233 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-host\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985254 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl68h\" (UniqueName: \"kubernetes.io/projected/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-kube-api-access-kl68h\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985275 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e87755-014a-4616-b966-b66c76c86721-host\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985295 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tvp2\" (UniqueName: \"kubernetes.io/projected/a0e87755-014a-4616-b966-b66c76c86721-kube-api-access-2tvp2\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985319 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s6st\" (UniqueName: \"kubernetes.io/projected/c4e5225b-1c04-42e3-93da-e3bef0b45c73-kube-api-access-5s6st\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985334 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985352 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-systemd\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985366 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-lib-modules\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985389 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysctl-conf\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985416 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0e87755-014a-4616-b966-b66c76c86721-serviceca\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985433 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27055683-09d8-4071-b297-5feb0eb4e3b8-tmp-dir\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985455 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-sys\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985495 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysconfig\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985522 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:38.985593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985551 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-registration-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985576 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-device-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985713 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-device-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985791 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysctl-d\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985795 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-lib-modules\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985840 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-host\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985849 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysctl-conf\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985902 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-socket-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985939 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-sys-fs\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.985991 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-run\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986025 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-var-lib-kubelet\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986082 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-etc-selinux\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986134 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-modprobe-d\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986174 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986169 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e87755-014a-4616-b966-b66c76c86721-host\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986243 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0e87755-014a-4616-b966-b66c76c86721-serviceca\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986282 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-sysconfig\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986353 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-sys\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.986453 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.986518 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret podName:0401d78e-6529-4987-90cb-4369375d67be nodeName:}" failed. No retries permitted until 2026-09-15 06:36:39.486500526 +0000 UTC m=+3.041157941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret") pod "global-pull-secret-syncer-96qh7" (UID: "0401d78e-6529-4987-90cb-4369375d67be") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986515 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27055683-09d8-4071-b297-5feb0eb4e3b8-hosts-file\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986515 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-registration-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986530 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-kubernetes\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986538 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27055683-09d8-4071-b297-5feb0eb4e3b8-tmp-dir\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986587 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-systemd\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986629 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4e5225b-1c04-42e3-93da-e3bef0b45c73-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986655 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0401d78e-6529-4987-90cb-4369375d67be-kubelet-config\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.986727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.986716 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0401d78e-6529-4987-90cb-4369375d67be-dbus\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:38.988953 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.988931 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-tmp\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.989073 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.988957 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-etc-tuned\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.996476 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.996423 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:38.996476 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.996454 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:38.996476 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.996469 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xvmc7 for pod openshift-network-diagnostics/network-check-target-pnhgk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:38.996678 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:38.996531 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7 podName:fa313485-4916-4bd9-9d9e-7d1a126a82a2 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:39.496514646 +0000 UTC m=+3.051172052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xvmc7" (UniqueName: "kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7") pod "network-check-target-pnhgk" (UID: "fa313485-4916-4bd9-9d9e-7d1a126a82a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:38.997090 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.997069 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl68h\" (UniqueName: \"kubernetes.io/projected/dc8cdf5d-8987-446a-ad13-4605b96a3e0d-kube-api-access-kl68h\") pod \"tuned-kqt7l\" (UID: \"dc8cdf5d-8987-446a-ad13-4605b96a3e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:38.998303 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.998280 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tvp2\" (UniqueName: \"kubernetes.io/projected/a0e87755-014a-4616-b966-b66c76c86721-kube-api-access-2tvp2\") pod \"node-ca-pjt59\" (UID: \"a0e87755-014a-4616-b966-b66c76c86721\") " pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:38.998660 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.998639 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9b78\" (UniqueName: \"kubernetes.io/projected/27055683-09d8-4071-b297-5feb0eb4e3b8-kube-api-access-j9b78\") pod \"node-resolver-m48nb\" (UID: \"27055683-09d8-4071-b297-5feb0eb4e3b8\") " pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:38.998755 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:38.998710 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s6st\" (UniqueName: \"kubernetes.io/projected/c4e5225b-1c04-42e3-93da-e3bef0b45c73-kube-api-access-5s6st\") pod \"aws-ebs-csi-driver-node-4r69w\" (UID: \"c4e5225b-1c04-42e3-93da-e3bef0b45c73\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:39.078809 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.078776 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z87dk" Sep 15 06:36:39.090761 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.090737 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6hddd" Sep 15 06:36:39.100639 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.100618 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:36:39.107572 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.107551 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qxgjj" Sep 15 06:36:39.114650 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.114631 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 15 06:36:39.115727 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.115709 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:39.123490 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.123468 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m48nb" Sep 15 06:36:39.131233 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.131211 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" Sep 15 06:36:39.140039 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.140019 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pjt59" Sep 15 06:36:39.144865 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.144845 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" Sep 15 06:36:39.390317 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.390232 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:39.390444 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:39.390405 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:39.390498 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:39.390468 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:36:40.390453848 +0000 UTC m=+3.945111248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:39.491249 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.491208 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:39.491416 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:39.491365 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:39.491472 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:39.491443 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret podName:0401d78e-6529-4987-90cb-4369375d67be nodeName:}" failed. No retries permitted until 2026-09-15 06:36:40.491422902 +0000 UTC m=+4.046080330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret") pod "global-pull-secret-syncer-96qh7" (UID: "0401d78e-6529-4987-90cb-4369375d67be") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:39.592403 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.592366 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:39.592575 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:39.592538 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:39.592575 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:39.592565 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:39.592661 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:39.592578 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xvmc7 for pod openshift-network-diagnostics/network-check-target-pnhgk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:39.592661 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:39.592635 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7 podName:fa313485-4916-4bd9-9d9e-7d1a126a82a2 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:40.592621276 +0000 UTC m=+4.147278694 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvmc7" (UniqueName: "kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7") pod "network-check-target-pnhgk" (UID: "fa313485-4916-4bd9-9d9e-7d1a126a82a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:39.816744 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.816637 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 06:31:38 +0000 UTC" deadline="2028-04-22 08:49:56.310789888 +0000 UTC" Sep 15 06:36:39.816744 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:39.816694 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14042h13m16.494099916s" Sep 15 06:36:40.060539 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:40.060514 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e07529d_1e9e_4024_8870_0400eb65c215.slice/crio-571151222f686c2e9ba9f789f19194ddb08a9c8c3ec83fe87ce745e73606618a WatchSource:0}: Error finding container 571151222f686c2e9ba9f789f19194ddb08a9c8c3ec83fe87ce745e73606618a: Status 404 returned error can't find the container with id 571151222f686c2e9ba9f789f19194ddb08a9c8c3ec83fe87ce745e73606618a Sep 15 06:36:40.063247 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:40.063213 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode48e916e_5d40_41bc_8359_bc216fd5b2dd.slice/crio-a0b09cece3ec1e885bcc690d7f6ecac60ae8ae4077bd1a71041947ea34274380 WatchSource:0}: Error finding container a0b09cece3ec1e885bcc690d7f6ecac60ae8ae4077bd1a71041947ea34274380: Status 404 returned error can't find the container with id a0b09cece3ec1e885bcc690d7f6ecac60ae8ae4077bd1a71041947ea34274380 Sep 15 06:36:40.067246 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:40.067224 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e87755_014a_4616_b966_b66c76c86721.slice/crio-868ce8bf1709e47b3c5b0daff49d76d55ab3e7b79c20585db5f3b40cbc4827f2 WatchSource:0}: Error finding container 868ce8bf1709e47b3c5b0daff49d76d55ab3e7b79c20585db5f3b40cbc4827f2: Status 404 returned error can't find the container with id 868ce8bf1709e47b3c5b0daff49d76d55ab3e7b79c20585db5f3b40cbc4827f2 Sep 15 06:36:40.068261 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:40.068213 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e5225b_1c04_42e3_93da_e3bef0b45c73.slice/crio-b51532934dd60ece59788ef71a734b2cd1bc39269de3d8c38f7ec40a47498ec2 WatchSource:0}: Error finding container b51532934dd60ece59788ef71a734b2cd1bc39269de3d8c38f7ec40a47498ec2: Status 404 returned error can't find the container with id b51532934dd60ece59788ef71a734b2cd1bc39269de3d8c38f7ec40a47498ec2 Sep 15 06:36:40.069314 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:40.069162 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae71f33_68fa_4077_8b65_38ff766cddc1.slice/crio-a372b256a788c225f32f2a18a5f4ac521199f1d420cfda1bc8f56ac336b1239a WatchSource:0}: Error finding container a372b256a788c225f32f2a18a5f4ac521199f1d420cfda1bc8f56ac336b1239a: Status 404 returned error can't find the container with id a372b256a788c225f32f2a18a5f4ac521199f1d420cfda1bc8f56ac336b1239a Sep 15 06:36:40.070025 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:40.069927 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc8cdf5d_8987_446a_ad13_4605b96a3e0d.slice/crio-5cfda6d2de144bb6f6520ea79232d04efc1fb31320502881110d3ac97bd7b056 WatchSource:0}: Error finding container 5cfda6d2de144bb6f6520ea79232d04efc1fb31320502881110d3ac97bd7b056: Status 404 returned error can't find the container with id 5cfda6d2de144bb6f6520ea79232d04efc1fb31320502881110d3ac97bd7b056 Sep 15 06:36:40.070871 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:40.070850 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178866a3_7693_4a4b_8a07_c489b6fed8a7.slice/crio-1097b642d75d43dd2e158d1487156eb7cb34db19fbf25a7fbb5b227dbb581d41 WatchSource:0}: Error finding container 1097b642d75d43dd2e158d1487156eb7cb34db19fbf25a7fbb5b227dbb581d41: Status 404 returned error can't find the container with id 1097b642d75d43dd2e158d1487156eb7cb34db19fbf25a7fbb5b227dbb581d41 Sep 15 06:36:40.071896 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:36:40.071876 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27055683_09d8_4071_b297_5feb0eb4e3b8.slice/crio-0a1c0ac2dc6279126ec1362e22bea1d068acdb19db0f965ee686abc5ffc014c1 WatchSource:0}: Error finding container 0a1c0ac2dc6279126ec1362e22bea1d068acdb19db0f965ee686abc5ffc014c1: Status 404 returned error can't find the container with id 0a1c0ac2dc6279126ec1362e22bea1d068acdb19db0f965ee686abc5ffc014c1 Sep 15 06:36:40.397123 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.397051 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:40.397249 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.397169 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:40.397249 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.397217 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:36:42.397203503 +0000 UTC m=+5.951860903 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:40.497794 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.497766 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:40.497947 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.497904 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:40.498001 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.497977 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret podName:0401d78e-6529-4987-90cb-4369375d67be nodeName:}" failed. No retries permitted until 2026-09-15 06:36:42.497948402 +0000 UTC m=+6.052605819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret") pod "global-pull-secret-syncer-96qh7" (UID: "0401d78e-6529-4987-90cb-4369375d67be") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:40.598183 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.598142 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:40.598357 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.598323 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:40.598357 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.598342 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:40.598357 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.598355 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xvmc7 for pod openshift-network-diagnostics/network-check-target-pnhgk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:40.598518 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.598411 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7 podName:fa313485-4916-4bd9-9d9e-7d1a126a82a2 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:42.59839264 +0000 UTC m=+6.153050057 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvmc7" (UniqueName: "kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7") pod "network-check-target-pnhgk" (UID: "fa313485-4916-4bd9-9d9e-7d1a126a82a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:40.817858 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.817736 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 06:31:38 +0000 UTC" deadline="2028-03-08 01:44:42.218578902 +0000 UTC" Sep 15 06:36:40.817858 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.817778 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12955h8m1.400807007s" Sep 15 06:36:40.857792 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.857091 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:40.857792 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.857227 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:40.857792 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.857630 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:40.857792 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.857723 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:40.858356 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.858217 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:40.858356 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:40.858310 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:40.876109 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.876076 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" event={"ID":"ed80a124298f3affca688f00871d8528","Type":"ContainerStarted","Data":"32c93a16efcbaec5083edad0b4653a2e5b8421b7cd0d54ee62d1fb1d92392868"} Sep 15 06:36:40.880533 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.880453 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m48nb" event={"ID":"27055683-09d8-4071-b297-5feb0eb4e3b8","Type":"ContainerStarted","Data":"0a1c0ac2dc6279126ec1362e22bea1d068acdb19db0f965ee686abc5ffc014c1"} Sep 15 06:36:40.885380 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.885296 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" event={"ID":"c4e5225b-1c04-42e3-93da-e3bef0b45c73","Type":"ContainerStarted","Data":"b51532934dd60ece59788ef71a734b2cd1bc39269de3d8c38f7ec40a47498ec2"} Sep 15 06:36:40.897077 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.896976 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"571151222f686c2e9ba9f789f19194ddb08a9c8c3ec83fe87ce745e73606618a"} Sep 15 06:36:40.899243 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.899181 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z87dk" event={"ID":"f694d292-e504-410d-924d-170bfeb5971c","Type":"ContainerStarted","Data":"39024c19987ee8dc91838ba07f0ead9cb65f60f69145c9ddb2cb68bc5a09f3d5"} Sep 15 06:36:40.911641 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.911410 2552 generic.go:356] "Generic (PLEG): container finished" podID="bce2d90bf9ede72f5f9236ac85ff5a6d" containerID="5ee9a0566a4e262193c1a16416176e60f35c1d2810a698103d58090cfbb80c50" exitCode=0 Sep 15 06:36:40.911641 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.911505 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" event={"ID":"bce2d90bf9ede72f5f9236ac85ff5a6d","Type":"ContainerDied","Data":"5ee9a0566a4e262193c1a16416176e60f35c1d2810a698103d58090cfbb80c50"} Sep 15 06:36:40.926121 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.926072 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-232.ec2.internal" podStartSLOduration=2.926048039 podStartE2EDuration="2.926048039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:40.890385794 +0000 UTC m=+4.445043218" watchObservedRunningTime="2026-09-15 06:36:40.926048039 +0000 UTC m=+4.480705464" Sep 15 06:36:40.934284 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.934256 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hddd" event={"ID":"178866a3-7693-4a4b-8a07-c489b6fed8a7","Type":"ContainerStarted","Data":"1097b642d75d43dd2e158d1487156eb7cb34db19fbf25a7fbb5b227dbb581d41"} Sep 15 06:36:40.946063 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.946038 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" event={"ID":"dc8cdf5d-8987-446a-ad13-4605b96a3e0d","Type":"ContainerStarted","Data":"5cfda6d2de144bb6f6520ea79232d04efc1fb31320502881110d3ac97bd7b056"} Sep 15 06:36:40.957777 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.957612 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vtltd" event={"ID":"8ae71f33-68fa-4077-8b65-38ff766cddc1","Type":"ContainerStarted","Data":"a372b256a788c225f32f2a18a5f4ac521199f1d420cfda1bc8f56ac336b1239a"} Sep 15 06:36:40.968771 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.968735 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pjt59" event={"ID":"a0e87755-014a-4616-b966-b66c76c86721","Type":"ContainerStarted","Data":"868ce8bf1709e47b3c5b0daff49d76d55ab3e7b79c20585db5f3b40cbc4827f2"} Sep 15 06:36:40.974611 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:40.974508 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qxgjj" event={"ID":"e48e916e-5d40-41bc-8359-bc216fd5b2dd","Type":"ContainerStarted","Data":"a0b09cece3ec1e885bcc690d7f6ecac60ae8ae4077bd1a71041947ea34274380"} Sep 15 06:36:41.989511 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:41.989015 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" event={"ID":"bce2d90bf9ede72f5f9236ac85ff5a6d","Type":"ContainerStarted","Data":"bfda88ff16ff18e598c9d99ca53148faf97b910080227ae260b57e8fa88a00f2"} Sep 15 06:36:42.004246 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:42.004196 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-232.ec2.internal" podStartSLOduration=4.004176034 podStartE2EDuration="4.004176034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:42.003650043 +0000 UTC m=+5.558307469" watchObservedRunningTime="2026-09-15 06:36:42.004176034 +0000 UTC m=+5.558833459" Sep 15 06:36:42.414330 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:42.414246 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:42.414490 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.414454 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:42.414557 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.414516 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:36:46.414498597 +0000 UTC m=+9.969155998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:42.515703 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:42.515664 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:42.515877 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.515841 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:42.515933 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.515901 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret podName:0401d78e-6529-4987-90cb-4369375d67be nodeName:}" failed. No retries permitted until 2026-09-15 06:36:46.515884459 +0000 UTC m=+10.070541863 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret") pod "global-pull-secret-syncer-96qh7" (UID: "0401d78e-6529-4987-90cb-4369375d67be") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:42.616841 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:42.616687 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:42.617044 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.616869 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:42.617044 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.616901 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:42.617044 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.616914 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xvmc7 for pod openshift-network-diagnostics/network-check-target-pnhgk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:42.617044 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.616991 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7 podName:fa313485-4916-4bd9-9d9e-7d1a126a82a2 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:46.616954229 +0000 UTC m=+10.171611655 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvmc7" (UniqueName: "kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7") pod "network-check-target-pnhgk" (UID: "fa313485-4916-4bd9-9d9e-7d1a126a82a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:42.856260 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:42.856114 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:42.856260 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:42.856132 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:42.856260 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:42.856140 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:42.856508 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.856264 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:42.856508 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.856404 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:42.856508 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:42.856489 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:44.857615 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:44.857092 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:44.857615 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:44.857128 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:44.857615 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:44.857092 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:44.857615 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:44.857226 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:44.857615 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:44.857312 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:44.857615 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:44.857398 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:46.448412 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:46.448376 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:46.448790 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.448563 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:46.448790 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.448641 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:36:54.448621206 +0000 UTC m=+18.003278622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:46.549634 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:46.549585 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:46.549812 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.549798 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:46.549872 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.549855 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret podName:0401d78e-6529-4987-90cb-4369375d67be nodeName:}" failed. No retries permitted until 2026-09-15 06:36:54.54984278 +0000 UTC m=+18.104500181 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret") pod "global-pull-secret-syncer-96qh7" (UID: "0401d78e-6529-4987-90cb-4369375d67be") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:46.650348 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:46.650299 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:46.650510 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.650481 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:46.650510 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.650500 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:46.650607 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.650513 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xvmc7 for pod openshift-network-diagnostics/network-check-target-pnhgk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:46.650607 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.650570 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7 podName:fa313485-4916-4bd9-9d9e-7d1a126a82a2 nodeName:}" failed. No retries permitted until 2026-09-15 06:36:54.65055276 +0000 UTC m=+18.205210185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvmc7" (UniqueName: "kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7") pod "network-check-target-pnhgk" (UID: "fa313485-4916-4bd9-9d9e-7d1a126a82a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:46.857358 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:46.856672 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:46.857358 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.856776 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:46.857358 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:46.857163 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:46.857358 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.857258 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:46.857916 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:46.857778 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:46.857916 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:46.857883 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:48.856386 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:48.856348 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:48.856885 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:48.856389 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:48.856885 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:48.856348 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:48.856885 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:48.856483 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:48.856885 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:48.856613 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:48.856885 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:48.856711 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:50.856072 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:50.856038 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:50.856433 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:50.856162 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:50.856433 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:50.856038 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:50.856433 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:50.856274 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:50.856433 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:50.856307 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:50.856433 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:50.856366 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:52.856345 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:52.856310 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:52.856784 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:52.856310 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:52.856784 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:52.856432 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:52.856784 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:52.856427 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:52.856784 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:52.856529 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:52.856784 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:52.856596 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:54.512232 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:54.512198 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:54.512713 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.512369 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:54.512713 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.512441 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:37:10.512419316 +0000 UTC m=+34.067076734 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:36:54.612940 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:54.612909 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:54.613127 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.613073 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:54.613182 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.613141 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret podName:0401d78e-6529-4987-90cb-4369375d67be nodeName:}" failed. No retries permitted until 2026-09-15 06:37:10.613120468 +0000 UTC m=+34.167777870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret") pod "global-pull-secret-syncer-96qh7" (UID: "0401d78e-6529-4987-90cb-4369375d67be") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:36:54.713637 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:54.713606 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:54.713804 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.713733 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:36:54.713804 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.713748 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:36:54.713804 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.713757 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xvmc7 for pod openshift-network-diagnostics/network-check-target-pnhgk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:54.713908 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.713808 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7 podName:fa313485-4916-4bd9-9d9e-7d1a126a82a2 nodeName:}" failed. No retries permitted until 2026-09-15 06:37:10.713792598 +0000 UTC m=+34.268450018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvmc7" (UniqueName: "kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7") pod "network-check-target-pnhgk" (UID: "fa313485-4916-4bd9-9d9e-7d1a126a82a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:36:54.856562 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:54.856480 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:54.856562 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:54.856518 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:54.856776 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.856621 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:54.856776 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:54.856641 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:54.856776 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.856765 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:54.856941 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:54.856844 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:56.856583 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:56.856552 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:56.857037 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:56.856704 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:56.857037 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:56.856715 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:56.857037 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:56.856809 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:56.857037 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:56.856870 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:56.857037 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:56.856949 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:58.036246 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.036041 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m48nb" event={"ID":"27055683-09d8-4071-b297-5feb0eb4e3b8","Type":"ContainerStarted","Data":"50603a7a7d8833b5001f07b08a3d84fffff39809430caa97748dd2a7d1b45f69"} Sep 15 06:36:58.038057 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.038032 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" event={"ID":"c4e5225b-1c04-42e3-93da-e3bef0b45c73","Type":"ContainerStarted","Data":"d07c3335e622625def749d2ff9aeae68edd6fb85372359213b01e472967add74"} Sep 15 06:36:58.040934 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.040904 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"1b421024dfb88b4fd0956e39b90ccd64c5372db66ba92ecd59325341be84ced2"} Sep 15 06:36:58.041023 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.041005 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"b919935805b81bf5377d5e5d388f9197db067c242a02e97045fcff01ce2971c5"} Sep 15 06:36:58.041080 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.041056 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"67b20e67dd426f3f41036c8ca32d0f14f14911adc0cf87f9a60bfe52413b5803"} Sep 15 06:36:58.041125 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.041103 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"799e2719118518cef77308d23cf54d2ea0440bbf65c2425b45d07e30cd86ae81"} Sep 15 06:36:58.042642 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.042608 2552 generic.go:356] "Generic (PLEG): container finished" podID="178866a3-7693-4a4b-8a07-c489b6fed8a7" containerID="177f24c9e35d12e7cd3a362f806b3f5762de4d01c896148c31c5af2b156d5f71" exitCode=0 Sep 15 06:36:58.042723 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.042640 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hddd" event={"ID":"178866a3-7693-4a4b-8a07-c489b6fed8a7","Type":"ContainerDied","Data":"177f24c9e35d12e7cd3a362f806b3f5762de4d01c896148c31c5af2b156d5f71"} Sep 15 06:36:58.045156 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.045132 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" event={"ID":"dc8cdf5d-8987-446a-ad13-4605b96a3e0d","Type":"ContainerStarted","Data":"1000487bbe9127efd101fe1f373f363f2eb5aa0b189cac1854c02aaabc45c1cc"} Sep 15 06:36:58.046634 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.046614 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vtltd" event={"ID":"8ae71f33-68fa-4077-8b65-38ff766cddc1","Type":"ContainerStarted","Data":"62d69c6ead547cbf1ca549d4a0827998335ba8bd2ab93ff88be7c23f88994992"} Sep 15 06:36:58.047942 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.047925 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pjt59" event={"ID":"a0e87755-014a-4616-b966-b66c76c86721","Type":"ContainerStarted","Data":"f0d63d8374c6fa1b0cab200b872f131b3a9203db5b54c2d53145f2b82ddb9882"} Sep 15 06:36:58.049219 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.049199 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qxgjj" event={"ID":"e48e916e-5d40-41bc-8359-bc216fd5b2dd","Type":"ContainerStarted","Data":"5b71df045ef0b9a0becf3e9c6a85c2e48f0df08f29fff69e768641c1653b5be1"} Sep 15 06:36:58.049718 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.049683 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m48nb" podStartSLOduration=4.486361603 podStartE2EDuration="22.049672263s" totalImagesPullingTime="17.56331066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:58.049430561 +0000 UTC m=+21.604088000" watchObservedRunningTime="2026-09-15 06:36:58.049672263 +0000 UTC m=+21.604329685" Sep 15 06:36:58.079836 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.079797 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vtltd" podStartSLOduration=8.505358871 podStartE2EDuration="22.079778479s" totalImagesPullingTime="13.574419608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:58.079321485 +0000 UTC m=+21.633978911" watchObservedRunningTime="2026-09-15 06:36:58.079778479 +0000 UTC m=+21.634435904" Sep 15 06:36:58.096242 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.096201 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kqt7l" podStartSLOduration=3.532375652 podStartE2EDuration="21.096191991s" totalImagesPullingTime="17.563816339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:58.095898143 +0000 UTC m=+21.650555560" watchObservedRunningTime="2026-09-15 06:36:58.096191991 +0000 UTC m=+21.650849413" Sep 15 06:36:58.111927 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.111889 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pjt59" podStartSLOduration=3.5552127540000003 podStartE2EDuration="21.111877461s" totalImagesPullingTime="17.556664707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:58.111350567 +0000 UTC m=+21.666007991" watchObservedRunningTime="2026-09-15 06:36:58.111877461 +0000 UTC m=+21.666534883" Sep 15 06:36:58.132555 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.132519 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qxgjj" podStartSLOduration=4.507352293 podStartE2EDuration="22.132506273s" totalImagesPullingTime="17.62515398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:58.132453148 +0000 UTC m=+21.687110571" watchObservedRunningTime="2026-09-15 06:36:58.132506273 +0000 UTC m=+21.687163695" Sep 15 06:36:58.856533 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.856354 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:36:58.856755 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.856362 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:36:58.856755 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:58.856610 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:36:58.856755 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:58.856393 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:36:58.856755 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:58.856677 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:36:58.856755 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:36:58.856738 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:36:59.055705 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.055671 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z87dk" event={"ID":"f694d292-e504-410d-924d-170bfeb5971c","Type":"ContainerStarted","Data":"194ee1a2dbddc092ac2322baffb7b10411dac5fcfe1f4bc7feaa4f0c2db9cf18"} Sep 15 06:36:59.058694 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.058665 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"5aaadbed109cbc964e0e28f8540a191cc722dc4ca0d805c878f84f7379ebbb13"} Sep 15 06:36:59.058815 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.058725 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"e1720de78fd80f42249b6739b1549953112e252fd2fc598ccdd9605c82c0f042"} Sep 15 06:36:59.069094 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.069061 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z87dk" podStartSLOduration=5.478227707 podStartE2EDuration="23.069049532s" totalImagesPullingTime="17.590821825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:36:59.068634227 +0000 UTC m=+22.623291672" watchObservedRunningTime="2026-09-15 06:36:59.069049532 +0000 UTC m=+22.623706954" Sep 15 06:36:59.263907 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.263859 2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 15 06:36:59.318329 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.318299 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:59.318932 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.318909 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:36:59.823629 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.823508 2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T06:36:59.263878642Z","UUID":"2f2ee65a-26a8-4b4c-ae62-8a4370b8bbd0","Handler":null,"Name":"","Endpoint":""} Sep 15 06:36:59.825344 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.825324 2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 15 06:36:59.825465 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:36:59.825354 2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 15 06:37:00.063816 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:00.063751 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" event={"ID":"c4e5225b-1c04-42e3-93da-e3bef0b45c73","Type":"ContainerStarted","Data":"d431cec27b9c930724469957bce9566b919e37f0b3584a0bf8d4bd8f166e1ed8"} Sep 15 06:37:00.064349 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:00.064330 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:37:00.064982 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:00.064944 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vtltd" Sep 15 06:37:00.856209 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:00.856125 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:00.856209 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:00.856131 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:00.856518 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:00.856246 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:37:00.856518 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:00.856263 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:37:00.856518 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:00.856348 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:37:00.856518 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:00.856445 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:37:01.068307 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:01.068273 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" event={"ID":"c4e5225b-1c04-42e3-93da-e3bef0b45c73","Type":"ContainerStarted","Data":"c2da41e9a726df3156f6f3f4e7a73e5cde26656cf51a82e32f3648cae6cae637"} Sep 15 06:37:01.071577 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:01.071553 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"846c14785c1aa5965618ed778c49eee19ca166e304269977dfdc711181f58449"} Sep 15 06:37:01.084675 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:01.084637 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4r69w" podStartSLOduration=3.976367395 podStartE2EDuration="24.08462343s" totalImagesPullingTime="20.108256035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:01.084361244 +0000 UTC m=+24.639018663" watchObservedRunningTime="2026-09-15 06:37:01.08462343 +0000 UTC m=+24.639280852" Sep 15 06:37:02.855761 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:02.855735 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:37:02.856346 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:02.855735 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:02.856346 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:02.855865 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:02.856346 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:02.855876 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:37:02.856346 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:02.855988 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:37:02.856346 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:02.856060 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:37:03.078381 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:03.078349 2552 generic.go:356] "Generic (PLEG): container finished" podID="178866a3-7693-4a4b-8a07-c489b6fed8a7" containerID="feb7b99a29e274579e0585408864cb0b4416955b78fb3364905e8c9f835bf6eb" exitCode=0 Sep 15 06:37:03.078523 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:03.078431 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hddd" event={"ID":"178866a3-7693-4a4b-8a07-c489b6fed8a7","Type":"ContainerDied","Data":"feb7b99a29e274579e0585408864cb0b4416955b78fb3364905e8c9f835bf6eb"} Sep 15 06:37:03.082316 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:03.082292 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" event={"ID":"4e07529d-1e9e-4024-8870-0400eb65c215","Type":"ContainerStarted","Data":"4ed1e0ae81540c3f5d8bd66c87d6d8113910f6a328c6568f731b7b9a8197b5ed"} Sep 15 06:37:03.082662 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:03.082633 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:37:03.097365 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:03.097345 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:37:03.121668 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:03.121626 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" podStartSLOduration=9.506589159 podStartE2EDuration="27.121611462s" totalImagesPullingTime="17.615022303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:03.121130747 +0000 UTC m=+26.675788169" watchObservedRunningTime="2026-09-15 06:37:03.121611462 +0000 UTC m=+26.676268885" Sep 15 06:37:04.086930 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.086850 2552 generic.go:356] "Generic (PLEG): container finished" podID="178866a3-7693-4a4b-8a07-c489b6fed8a7" containerID="dba54c3e573331437dba5179b24a2d3294fc9fd89a67f4d68cacebbbc8cbc1f8" exitCode=0 Sep 15 06:37:04.086930 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.086917 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hddd" event={"ID":"178866a3-7693-4a4b-8a07-c489b6fed8a7","Type":"ContainerDied","Data":"dba54c3e573331437dba5179b24a2d3294fc9fd89a67f4d68cacebbbc8cbc1f8"} Sep 15 06:37:04.090312 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.089590 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:37:04.090312 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.089713 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:37:04.104916 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.104890 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:37:04.550635 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.550444 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-96qh7"] Sep 15 06:37:04.550825 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.550746 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:04.550907 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:04.550848 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:37:04.553802 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.553766 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pnhgk"] Sep 15 06:37:04.553905 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.553882 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:04.554034 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:04.554012 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:37:04.565117 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.565089 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ns6cw"] Sep 15 06:37:04.565215 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:04.565173 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:37:04.565279 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:04.565245 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:37:05.856005 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:05.855959 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:05.856390 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:05.856110 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:37:06.856613 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:06.856584 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:06.857046 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:06.856684 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:37:06.857046 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:06.856723 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:37:06.857046 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:06.856804 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:37:07.095158 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:07.095070 2552 generic.go:356] "Generic (PLEG): container finished" podID="178866a3-7693-4a4b-8a07-c489b6fed8a7" containerID="68849f83d14cfb0f8b032bc95fe3b8342dc999ac3750831b410fcced03dfed5d" exitCode=0 Sep 15 06:37:07.095158 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:07.095138 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hddd" event={"ID":"178866a3-7693-4a4b-8a07-c489b6fed8a7","Type":"ContainerDied","Data":"68849f83d14cfb0f8b032bc95fe3b8342dc999ac3750831b410fcced03dfed5d"} Sep 15 06:37:07.856034 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:07.856007 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:07.856204 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:07.856119 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:37:08.855840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:08.855810 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:08.856315 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:08.855928 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96qh7" podUID="0401d78e-6529-4987-90cb-4369375d67be" Sep 15 06:37:08.856315 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:08.856025 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:37:08.856315 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:08.856124 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:37:09.856071 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:09.856042 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:09.856574 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:09.856149 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pnhgk" podUID="fa313485-4916-4bd9-9d9e-7d1a126a82a2" Sep 15 06:37:10.254930 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.254725 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-232.ec2.internal" event="NodeReady" Sep 15 06:37:10.255103 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.255055 2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 15 06:37:10.296846 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.296810 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6nc7q"] Sep 15 06:37:10.334295 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.334261 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6dtk4"] Sep 15 06:37:10.334458 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.334389 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.337179 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.337153 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 15 06:37:10.337308 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.337182 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k55p\"" Sep 15 06:37:10.337308 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.337159 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 15 06:37:10.349671 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.349652 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6nc7q"] Sep 15 06:37:10.349789 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.349694 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6dtk4"] Sep 15 06:37:10.349789 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.349763 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6dtk4" Sep 15 06:37:10.352960 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.352940 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 15 06:37:10.353091 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.352985 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 15 06:37:10.353091 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.353001 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 15 06:37:10.353190 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.353103 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t5mx2\"" Sep 15 06:37:10.424130 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.424091 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-config-volume\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.424299 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.424166 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.424299 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.424200 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-924gd\" (UniqueName: \"kubernetes.io/projected/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-kube-api-access-924gd\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.424299 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.424218 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-tmp-dir\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.524914 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.524877 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:37:10.524914 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.524918 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-config-volume\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.525162 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.524961 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.525162 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.524998 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8dda479-a879-433e-b4ca-b667b60c8c01-cert\") pod \"ingress-canary-6dtk4\" (UID: \"f8dda479-a879-433e-b4ca-b667b60c8c01\") " pod="openshift-ingress-canary/ingress-canary-6dtk4" Sep 15 06:37:10.525162 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.525025 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-924gd\" (UniqueName: \"kubernetes.io/projected/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-kube-api-access-924gd\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.525162 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.525051 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-tmp-dir\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.525162 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.525063 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:37:10.525162 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.525097 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp246\" (UniqueName: \"kubernetes.io/projected/f8dda479-a879-433e-b4ca-b667b60c8c01-kube-api-access-cp246\") pod \"ingress-canary-6dtk4\" (UID: \"f8dda479-a879-433e-b4ca-b667b60c8c01\") " pod="openshift-ingress-canary/ingress-canary-6dtk4" Sep 15 06:37:10.525162 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.525116 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:10.525162 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.525153 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:37:42.52513061 +0000 UTC m=+66.079788028 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 15 06:37:10.525480 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.525204 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:37:11.025190858 +0000 UTC m=+34.579848286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:37:10.525480 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.525445 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-tmp-dir\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.525620 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.525596 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-config-volume\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.534953 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.534926 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-924gd\" (UniqueName: \"kubernetes.io/projected/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-kube-api-access-924gd\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:10.625838 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.625800 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8dda479-a879-433e-b4ca-b667b60c8c01-cert\") pod \"ingress-canary-6dtk4\" (UID: \"f8dda479-a879-433e-b4ca-b667b60c8c01\") " pod="openshift-ingress-canary/ingress-canary-6dtk4" Sep 15 06:37:10.626013 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.625860 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:10.626013 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.625893 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp246\" (UniqueName: \"kubernetes.io/projected/f8dda479-a879-433e-b4ca-b667b60c8c01-kube-api-access-cp246\") pod \"ingress-canary-6dtk4\" (UID: \"f8dda479-a879-433e-b4ca-b667b60c8c01\") " pod="openshift-ingress-canary/ingress-canary-6dtk4" Sep 15 06:37:10.626276 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.626244 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 15 06:37:10.626383 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.626334 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret podName:0401d78e-6529-4987-90cb-4369375d67be nodeName:}" failed. No retries permitted until 2026-09-15 06:37:42.626314461 +0000 UTC m=+66.180971870 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret") pod "global-pull-secret-syncer-96qh7" (UID: "0401d78e-6529-4987-90cb-4369375d67be") : object "kube-system"/"original-pull-secret" not registered Sep 15 06:37:10.628571 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.628544 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8dda479-a879-433e-b4ca-b667b60c8c01-cert\") pod \"ingress-canary-6dtk4\" (UID: \"f8dda479-a879-433e-b4ca-b667b60c8c01\") " pod="openshift-ingress-canary/ingress-canary-6dtk4" Sep 15 06:37:10.634000 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.633960 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp246\" (UniqueName: \"kubernetes.io/projected/f8dda479-a879-433e-b4ca-b667b60c8c01-kube-api-access-cp246\") pod \"ingress-canary-6dtk4\" (UID: \"f8dda479-a879-433e-b4ca-b667b60c8c01\") " pod="openshift-ingress-canary/ingress-canary-6dtk4" Sep 15 06:37:10.658947 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.658922 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6dtk4" Sep 15 06:37:10.726604 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.726574 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:10.726777 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.726723 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 15 06:37:10.726777 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.726742 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 15 06:37:10.726777 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.726751 2552 projected.go:196] Error preparing data for projected volume kube-api-access-xvmc7 for pod openshift-network-diagnostics/network-check-target-pnhgk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:37:10.726902 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:10.726803 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7 podName:fa313485-4916-4bd9-9d9e-7d1a126a82a2 nodeName:}" failed. No retries permitted until 2026-09-15 06:37:42.726786567 +0000 UTC m=+66.281443969 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xvmc7" (UniqueName: "kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7") pod "network-check-target-pnhgk" (UID: "fa313485-4916-4bd9-9d9e-7d1a126a82a2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 15 06:37:10.856663 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.856625 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:37:10.857134 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.856820 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:10.859482 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.859409 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 06:37:10.859482 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.859434 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5bkjm\"" Sep 15 06:37:10.859482 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:10.859413 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 06:37:11.028841 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:11.028812 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:11.028994 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:11.028979 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:11.029061 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:11.029049 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:37:12.029034054 +0000 UTC m=+35.583691454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:37:11.275040 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:11.274956 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6dtk4"] Sep 15 06:37:11.339283 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:37:11.339255 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8dda479_a879_433e_b4ca_b667b60c8c01.slice/crio-fa6dbb2c63b4e226fd6a7036941b2c2eca54d05d34a407695ad758caadcb51d7 WatchSource:0}: Error finding container fa6dbb2c63b4e226fd6a7036941b2c2eca54d05d34a407695ad758caadcb51d7: Status 404 returned error can't find the container with id fa6dbb2c63b4e226fd6a7036941b2c2eca54d05d34a407695ad758caadcb51d7 Sep 15 06:37:11.856319 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:11.856083 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:11.858940 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:11.858917 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 06:37:11.859514 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:11.859022 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9fzgn\"" Sep 15 06:37:11.859514 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:11.859022 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 06:37:12.036110 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:12.036077 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:12.036268 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:12.036237 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:12.036314 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:12.036308 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:37:14.036284679 +0000 UTC m=+37.590942081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:37:12.109989 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:12.109895 2552 generic.go:356] "Generic (PLEG): container finished" podID="178866a3-7693-4a4b-8a07-c489b6fed8a7" containerID="b61d3d995b3c63da7dde7070833d17aee60f565f5c3c46d727c2fd41d433c811" exitCode=0 Sep 15 06:37:12.110156 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:12.109987 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hddd" event={"ID":"178866a3-7693-4a4b-8a07-c489b6fed8a7","Type":"ContainerDied","Data":"b61d3d995b3c63da7dde7070833d17aee60f565f5c3c46d727c2fd41d433c811"} Sep 15 06:37:12.111101 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:12.111079 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6dtk4" event={"ID":"f8dda479-a879-433e-b4ca-b667b60c8c01","Type":"ContainerStarted","Data":"fa6dbb2c63b4e226fd6a7036941b2c2eca54d05d34a407695ad758caadcb51d7"} Sep 15 06:37:13.116628 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:13.116597 2552 generic.go:356] "Generic (PLEG): container finished" podID="178866a3-7693-4a4b-8a07-c489b6fed8a7" containerID="588c6cff92d3bcb7235d4d74eef2990661f6db25a0d576939a2d4b0ea06cc854" exitCode=0 Sep 15 06:37:13.117011 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:13.116641 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hddd" event={"ID":"178866a3-7693-4a4b-8a07-c489b6fed8a7","Type":"ContainerDied","Data":"588c6cff92d3bcb7235d4d74eef2990661f6db25a0d576939a2d4b0ea06cc854"} Sep 15 06:37:14.050840 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:14.050798 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:14.051063 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:14.050924 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:14.051063 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:14.050993 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:37:18.050980538 +0000 UTC m=+41.605637952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:37:14.123343 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:14.123299 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6hddd" event={"ID":"178866a3-7693-4a4b-8a07-c489b6fed8a7","Type":"ContainerStarted","Data":"8955ab7e4d15d6039e07e01a747eba7957ac83e744e69c9d2c6bedb7f18e6018"} Sep 15 06:37:14.144380 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:14.144331 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6hddd" podStartSLOduration=8.700183581 podStartE2EDuration="38.144308674s" totalImagesPullingTime="29.444125093s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:14.143953907 +0000 UTC m=+37.698611354" watchObservedRunningTime="2026-09-15 06:37:14.144308674 +0000 UTC m=+37.698966098" Sep 15 06:37:16.130354 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:16.130309 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6dtk4" event={"ID":"f8dda479-a879-433e-b4ca-b667b60c8c01","Type":"ContainerStarted","Data":"9248829564ae16181a5dfd589a0117899feec4284a16608597c4b97c3552a4bb"} Sep 15 06:37:16.151149 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:16.149267 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6dtk4" podStartSLOduration=1.954568251 podStartE2EDuration="6.149247653s" totalImagesPullingTime="4.194679402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:16.147879689 +0000 UTC m=+39.702537111" watchObservedRunningTime="2026-09-15 06:37:16.149247653 +0000 UTC m=+39.703905078" Sep 15 06:37:18.077671 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:18.077636 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:18.078037 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:18.077789 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:18.078037 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:18.077853 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:37:26.077837317 +0000 UTC m=+49.632494738 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:37:26.131884 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:26.131841 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:26.132327 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:26.131997 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:26.132327 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:26.132067 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:37:42.132052569 +0000 UTC m=+65.686709974 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:37:36.101245 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:36.101214 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgvr8" Sep 15 06:37:42.136662 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.136627 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:37:42.137129 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:42.136758 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:37:42.137129 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:42.136825 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:38:14.136806733 +0000 UTC m=+97.691464135 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:37:42.538771 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.538735 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:37:42.541455 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.541436 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 15 06:37:42.549708 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:42.549692 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 15 06:37:42.549766 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:37:42.549756 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:38:46.549737998 +0000 UTC m=+130.104395399 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : secret "metrics-daemon-secret" not found Sep 15 06:37:42.640029 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.640004 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:42.642947 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.642929 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 15 06:37:42.654132 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.654114 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0401d78e-6529-4987-90cb-4369375d67be-original-pull-secret\") pod \"global-pull-secret-syncer-96qh7\" (UID: \"0401d78e-6529-4987-90cb-4369375d67be\") " pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:42.674298 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.674277 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96qh7" Sep 15 06:37:42.740717 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.740684 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:42.743449 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.743431 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 15 06:37:42.753787 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.753771 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 15 06:37:42.764647 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.764625 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvmc7\" (UniqueName: \"kubernetes.io/projected/fa313485-4916-4bd9-9d9e-7d1a126a82a2-kube-api-access-xvmc7\") pod \"network-check-target-pnhgk\" (UID: \"fa313485-4916-4bd9-9d9e-7d1a126a82a2\") " pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:42.767437 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.767416 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9fzgn\"" Sep 15 06:37:42.775615 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.775600 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:42.939826 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:42.939796 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-96qh7"] Sep 15 06:37:42.942573 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:37:42.942546 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0401d78e_6529_4987_90cb_4369375d67be.slice/crio-7b12a92548364406115bf268ca56c8f3a6bb2b23bc7fcd0f82a5812be67c2b04 WatchSource:0}: Error finding container 7b12a92548364406115bf268ca56c8f3a6bb2b23bc7fcd0f82a5812be67c2b04: Status 404 returned error can't find the container with id 7b12a92548364406115bf268ca56c8f3a6bb2b23bc7fcd0f82a5812be67c2b04 Sep 15 06:37:43.036798 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:43.036773 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pnhgk"] Sep 15 06:37:43.039607 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:37:43.039585 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa313485_4916_4bd9_9d9e_7d1a126a82a2.slice/crio-34da7930a5460e614b63209bc026d0b9fa66094a46e6815cb93f50b5cb531b5c WatchSource:0}: Error finding container 34da7930a5460e614b63209bc026d0b9fa66094a46e6815cb93f50b5cb531b5c: Status 404 returned error can't find the container with id 34da7930a5460e614b63209bc026d0b9fa66094a46e6815cb93f50b5cb531b5c Sep 15 06:37:43.190622 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:43.190585 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pnhgk" event={"ID":"fa313485-4916-4bd9-9d9e-7d1a126a82a2","Type":"ContainerStarted","Data":"34da7930a5460e614b63209bc026d0b9fa66094a46e6815cb93f50b5cb531b5c"} Sep 15 06:37:43.191747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:43.191723 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-96qh7" event={"ID":"0401d78e-6529-4987-90cb-4369375d67be","Type":"ContainerStarted","Data":"7b12a92548364406115bf268ca56c8f3a6bb2b23bc7fcd0f82a5812be67c2b04"} Sep 15 06:37:46.202886 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:46.202848 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pnhgk" event={"ID":"fa313485-4916-4bd9-9d9e-7d1a126a82a2","Type":"ContainerStarted","Data":"314e97142c01519f5238ef34a3014bda81a2a48c1b5cd1cb17d33921b07a02d8"} Sep 15 06:37:46.203316 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:46.203007 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:37:46.218045 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:46.217987 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pnhgk" podStartSLOduration=67.342643682 podStartE2EDuration="1m10.21795333s" totalImagesPullingTime="2.875309648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:46.217406153 +0000 UTC m=+69.772063576" watchObservedRunningTime="2026-09-15 06:37:46.21795333 +0000 UTC m=+69.772610752" Sep 15 06:37:48.211951 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:48.211911 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-96qh7" event={"ID":"0401d78e-6529-4987-90cb-4369375d67be","Type":"ContainerStarted","Data":"bfddcc3a92351f44eac40c2c89f19bea68f9f71c0986c3dfc2801ce921147ad7"} Sep 15 06:37:48.227005 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:37:48.226940 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-96qh7" podStartSLOduration=66.663446502 podStartE2EDuration="1m11.226926492s" totalImagesPullingTime="4.56347999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:37:48.225937853 +0000 UTC m=+71.780595276" watchObservedRunningTime="2026-09-15 06:37:48.226926492 +0000 UTC m=+71.781583914" Sep 15 06:38:14.154184 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:14.154143 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:38:14.154614 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:38:14.154258 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:38:14.154614 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:38:14.154310 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:39:18.154297543 +0000 UTC m=+161.708954944 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:38:17.208526 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:17.208498 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pnhgk" Sep 15 06:38:46.580528 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:46.580494 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:38:46.581023 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:38:46.580619 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 15 06:38:46.581023 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:38:46.580688 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:40:48.580674007 +0000 UTC m=+252.135331407 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : secret "metrics-daemon-secret" not found Sep 15 06:38:59.316881 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.316850 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27"] Sep 15 06:38:59.318684 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.318667 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.319935 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.319912 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k"] Sep 15 06:38:59.321387 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.321365 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 15 06:38:59.321523 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.321504 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:38:59.321601 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.321502 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 15 06:38:59.322768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.322753 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 15 06:38:59.322768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.322759 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 15 06:38:59.324190 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.324171 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6l4m9\"" Sep 15 06:38:59.324310 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.324209 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 15 06:38:59.333017 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.332951 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27"] Sep 15 06:38:59.333615 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.333596 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k"] Sep 15 06:38:59.460731 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.460704 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-tmp\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.460866 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.460742 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.460866 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.460765 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5f74\" (UniqueName: \"kubernetes.io/projected/443aa338-2090-4ef7-8075-920e3cc8de4d-kube-api-access-n5f74\") pod \"managed-serviceaccount-addon-agent-85d88bdf64-zkm7k\" (UID: \"443aa338-2090-4ef7-8075-920e3cc8de4d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:38:59.460866 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.460783 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh42f\" (UniqueName: \"kubernetes.io/projected/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-kube-api-access-zh42f\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.460866 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.460810 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/443aa338-2090-4ef7-8075-920e3cc8de4d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85d88bdf64-zkm7k\" (UID: \"443aa338-2090-4ef7-8075-920e3cc8de4d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:38:59.561790 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.561766 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-tmp\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.561878 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.561806 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.561878 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.561835 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5f74\" (UniqueName: \"kubernetes.io/projected/443aa338-2090-4ef7-8075-920e3cc8de4d-kube-api-access-n5f74\") pod \"managed-serviceaccount-addon-agent-85d88bdf64-zkm7k\" (UID: \"443aa338-2090-4ef7-8075-920e3cc8de4d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:38:59.561878 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.561860 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh42f\" (UniqueName: \"kubernetes.io/projected/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-kube-api-access-zh42f\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.562004 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.561888 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/443aa338-2090-4ef7-8075-920e3cc8de4d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85d88bdf64-zkm7k\" (UID: \"443aa338-2090-4ef7-8075-920e3cc8de4d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:38:59.562237 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.562214 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-tmp\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.564339 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.564322 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/443aa338-2090-4ef7-8075-920e3cc8de4d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85d88bdf64-zkm7k\" (UID: \"443aa338-2090-4ef7-8075-920e3cc8de4d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:38:59.564519 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.564500 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.569612 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.569568 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh42f\" (UniqueName: \"kubernetes.io/projected/a8c247d7-28c7-4fe8-8c34-f3ad56d96110-kube-api-access-zh42f\") pod \"klusterlet-addon-workmgr-5c678594fd-ldc27\" (UID: \"a8c247d7-28c7-4fe8-8c34-f3ad56d96110\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.569682 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.569613 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5f74\" (UniqueName: \"kubernetes.io/projected/443aa338-2090-4ef7-8075-920e3cc8de4d-kube-api-access-n5f74\") pod \"managed-serviceaccount-addon-agent-85d88bdf64-zkm7k\" (UID: \"443aa338-2090-4ef7-8075-920e3cc8de4d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:38:59.628725 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.628704 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:38:59.648521 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.648495 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:38:59.895883 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.895857 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27"] Sep 15 06:38:59.898276 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:38:59.898248 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c247d7_28c7_4fe8_8c34_f3ad56d96110.slice/crio-6c7f6fe4c663458cb9f994cc050c8e7f10321118afc8f67758fca9dd9362ba01 WatchSource:0}: Error finding container 6c7f6fe4c663458cb9f994cc050c8e7f10321118afc8f67758fca9dd9362ba01: Status 404 returned error can't find the container with id 6c7f6fe4c663458cb9f994cc050c8e7f10321118afc8f67758fca9dd9362ba01 Sep 15 06:38:59.918113 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:38:59.918092 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k"] Sep 15 06:38:59.920041 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:38:59.920016 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod443aa338_2090_4ef7_8075_920e3cc8de4d.slice/crio-5d2b23679816e3182da7e90986e3c8a46bb277f213520bb746a559b6f99b8aca WatchSource:0}: Error finding container 5d2b23679816e3182da7e90986e3c8a46bb277f213520bb746a559b6f99b8aca: Status 404 returned error can't find the container with id 5d2b23679816e3182da7e90986e3c8a46bb277f213520bb746a559b6f99b8aca Sep 15 06:39:00.382047 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:00.382003 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" event={"ID":"443aa338-2090-4ef7-8075-920e3cc8de4d","Type":"ContainerStarted","Data":"5d2b23679816e3182da7e90986e3c8a46bb277f213520bb746a559b6f99b8aca"} Sep 15 06:39:00.383107 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:00.383081 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" event={"ID":"a8c247d7-28c7-4fe8-8c34-f3ad56d96110","Type":"ContainerStarted","Data":"6c7f6fe4c663458cb9f994cc050c8e7f10321118afc8f67758fca9dd9362ba01"} Sep 15 06:39:04.395627 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:04.395592 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" event={"ID":"a8c247d7-28c7-4fe8-8c34-f3ad56d96110","Type":"ContainerStarted","Data":"d4d55eca92a2f6dd22ac3a03408e85b4e9db0531a0d8d835ad01935a070583d1"} Sep 15 06:39:04.396110 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:04.395818 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:39:04.397472 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:04.397451 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:39:04.412527 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:04.412485 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" podStartSLOduration=1.519181084 podStartE2EDuration="5.412473022s" totalImagesPullingTime="3.893291938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:39:04.411124989 +0000 UTC m=+147.965782422" watchObservedRunningTime="2026-09-15 06:39:04.412473022 +0000 UTC m=+147.967130465" Sep 15 06:39:08.407898 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:08.407861 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" event={"ID":"443aa338-2090-4ef7-8075-920e3cc8de4d","Type":"ContainerStarted","Data":"3c1eb1d4da1085547d191363b83e3aa99f89115a47623276085a69a836130df2"} Sep 15 06:39:08.423710 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:08.423656 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" podStartSLOduration=1.651563385 podStartE2EDuration="9.423641911s" totalImagesPullingTime="7.772078526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:39:08.422767698 +0000 UTC m=+151.977425134" watchObservedRunningTime="2026-09-15 06:39:08.423641911 +0000 UTC m=+151.978299333" Sep 15 06:39:13.345135 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:39:13.345092 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6nc7q" podUID="93f7caf1-70fc-43c2-b8f9-9710fbbf2cea" Sep 15 06:39:13.422689 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:13.422653 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6nc7q" Sep 15 06:39:13.869069 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:39:13.869019 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:39:18.193403 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:18.193352 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:39:18.193864 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:39:18.193496 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 15 06:39:18.193864 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:39:18.193573 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls podName:93f7caf1-70fc-43c2-b8f9-9710fbbf2cea nodeName:}" failed. No retries permitted until 2026-09-15 06:41:20.193556415 +0000 UTC m=+283.748213816 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls") pod "dns-default-6nc7q" (UID: "93f7caf1-70fc-43c2-b8f9-9710fbbf2cea") : secret "dns-default-metrics-tls" not found Sep 15 06:39:28.855759 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:39:28.855720 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:40:46.116644 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.116607 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cfbb448f7-8d6tr"] Sep 15 06:40:46.119358 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.119343 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.123091 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.123064 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 15 06:40:46.123213 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.123161 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4fjls\"" Sep 15 06:40:46.124074 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.124057 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 15 06:40:46.124323 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.124300 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 15 06:40:46.129208 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.129177 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 15 06:40:46.136148 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.136125 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cfbb448f7-8d6tr"] Sep 15 06:40:46.209897 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.209871 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m"] Sep 15 06:40:46.212593 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.212575 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.215508 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.215485 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 15 06:40:46.216877 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.216860 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sqqlz\"" Sep 15 06:40:46.216977 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.216939 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 15 06:40:46.216977 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.216954 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 15 06:40:46.217086 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.216960 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 15 06:40:46.230695 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.230677 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m"] Sep 15 06:40:46.288200 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288173 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/82de95cc-b9c4-4546-b870-588edc3da907-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.288307 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288205 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3cb1e519-f962-473d-837e-b4c98befa2ee-ca-trust-extracted\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.288307 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288237 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-installation-pull-secrets\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.288307 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288262 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-trusted-ca\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.288424 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288316 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-bound-sa-token\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.288424 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288348 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-image-registry-private-configuration\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.288424 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288369 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.288424 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288399 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.288424 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288417 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsgg4\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-kube-api-access-bsgg4\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.288611 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288442 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djx5z\" (UniqueName: \"kubernetes.io/projected/82de95cc-b9c4-4546-b870-588edc3da907-kube-api-access-djx5z\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.288611 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.288459 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-certificates\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.389572 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389494 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-image-registry-private-configuration\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.389572 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389531 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.389572 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389563 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389581 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsgg4\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-kube-api-access-bsgg4\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.389659 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.389680 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfbb448f7-8d6tr: secret "image-registry-tls" not found Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389690 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djx5z\" (UniqueName: \"kubernetes.io/projected/82de95cc-b9c4-4546-b870-588edc3da907-kube-api-access-djx5z\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.389723 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls podName:3cb1e519-f962-473d-837e-b4c98befa2ee nodeName:}" failed. No retries permitted until 2026-09-15 06:40:46.889709583 +0000 UTC m=+250.444367011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls") pod "image-registry-5cfbb448f7-8d6tr" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee") : secret "image-registry-tls" not found Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389738 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-certificates\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.389661 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389770 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/82de95cc-b9c4-4546-b870-588edc3da907-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.389805 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389794 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3cb1e519-f962-473d-837e-b4c98befa2ee-ca-trust-extracted\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.390274 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.389830 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls podName:82de95cc-b9c4-4546-b870-588edc3da907 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:46.88981049 +0000 UTC m=+250.444467899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-2tg7m" (UID: "82de95cc-b9c4-4546-b870-588edc3da907") : secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:46.390274 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389859 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-installation-pull-secrets\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.390274 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389887 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-trusted-ca\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.390274 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.389919 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-bound-sa-token\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.390274 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.390185 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3cb1e519-f962-473d-837e-b4c98befa2ee-ca-trust-extracted\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.390538 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.390513 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/82de95cc-b9c4-4546-b870-588edc3da907-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.390827 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.390803 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-certificates\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.390986 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.390951 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-trusted-ca\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.392289 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.392265 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-installation-pull-secrets\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.392373 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.392265 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-image-registry-private-configuration\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.397933 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.397913 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-bound-sa-token\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.398309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.398290 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsgg4\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-kube-api-access-bsgg4\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.398309 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.398301 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djx5z\" (UniqueName: \"kubernetes.io/projected/82de95cc-b9c4-4546-b870-588edc3da907-kube-api-access-djx5z\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.894248 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.894213 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:46.894419 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:46.894258 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:46.894419 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.894355 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:46.894419 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.894360 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 06:40:46.894419 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.894375 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfbb448f7-8d6tr: secret "image-registry-tls" not found Sep 15 06:40:46.894556 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.894424 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls podName:82de95cc-b9c4-4546-b870-588edc3da907 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:47.894408222 +0000 UTC m=+251.449065643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-2tg7m" (UID: "82de95cc-b9c4-4546-b870-588edc3da907") : secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:46.894556 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:46.894438 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls podName:3cb1e519-f962-473d-837e-b4c98befa2ee nodeName:}" failed. No retries permitted until 2026-09-15 06:40:47.894431276 +0000 UTC m=+251.449088676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls") pod "image-registry-5cfbb448f7-8d6tr" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee") : secret "image-registry-tls" not found Sep 15 06:40:47.902569 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:47.902537 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:47.902984 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:47.902607 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:47.902984 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:47.902694 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 06:40:47.902984 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:47.902705 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfbb448f7-8d6tr: secret "image-registry-tls" not found Sep 15 06:40:47.902984 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:47.902710 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:47.902984 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:47.902768 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls podName:3cb1e519-f962-473d-837e-b4c98befa2ee nodeName:}" failed. No retries permitted until 2026-09-15 06:40:49.902752869 +0000 UTC m=+253.457410269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls") pod "image-registry-5cfbb448f7-8d6tr" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee") : secret "image-registry-tls" not found Sep 15 06:40:47.902984 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:47.902781 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls podName:82de95cc-b9c4-4546-b870-588edc3da907 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:49.902775267 +0000 UTC m=+253.457432668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-2tg7m" (UID: "82de95cc-b9c4-4546-b870-588edc3da907") : secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:48.606217 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:48.606171 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:40:48.606399 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:48.606323 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 15 06:40:48.606399 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:48.606384 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs podName:532a12f1-78ac-4e4d-a02d-6be6c808825f nodeName:}" failed. No retries permitted until 2026-09-15 06:42:50.60637033 +0000 UTC m=+374.161027731 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs") pod "network-metrics-daemon-ns6cw" (UID: "532a12f1-78ac-4e4d-a02d-6be6c808825f") : secret "metrics-daemon-secret" not found Sep 15 06:40:49.915649 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:49.915610 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:49.916101 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:49.915663 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:49.916101 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:49.915756 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 06:40:49.916101 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:49.915775 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfbb448f7-8d6tr: secret "image-registry-tls" not found Sep 15 06:40:49.916101 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:49.915808 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:49.916101 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:49.915828 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls podName:3cb1e519-f962-473d-837e-b4c98befa2ee nodeName:}" failed. No retries permitted until 2026-09-15 06:40:53.915814003 +0000 UTC m=+257.470471403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls") pod "image-registry-5cfbb448f7-8d6tr" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee") : secret "image-registry-tls" not found Sep 15 06:40:49.916101 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:49.915859 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls podName:82de95cc-b9c4-4546-b870-588edc3da907 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:53.915844256 +0000 UTC m=+257.470501658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-2tg7m" (UID: "82de95cc-b9c4-4546-b870-588edc3da907") : secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:51.755822 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.755785 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-84cfd"] Sep 15 06:40:51.759818 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.759802 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:51.762424 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.762397 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2dx68\"" Sep 15 06:40:51.762561 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.762495 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 15 06:40:51.762561 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.762497 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 15 06:40:51.763656 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.763636 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 15 06:40:51.763738 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.763675 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 15 06:40:51.766128 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.766107 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-84cfd"] Sep 15 06:40:51.931690 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.931649 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:51.931690 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.931692 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d9de86b-d55c-49ef-83a8-677097aa9444-data-volume\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:51.931891 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.931712 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6m68\" (UniqueName: \"kubernetes.io/projected/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-api-access-p6m68\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:51.931891 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.931743 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d9de86b-d55c-49ef-83a8-677097aa9444-crio-socket\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:51.931891 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:51.931793 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.032860 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.032772 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.033023 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.032868 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.033023 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:52.032894 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:52.033023 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.032908 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d9de86b-d55c-49ef-83a8-677097aa9444-data-volume\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.033023 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.032924 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6m68\" (UniqueName: \"kubernetes.io/projected/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-api-access-p6m68\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.033023 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:52.032943 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:52.532929463 +0000 UTC m=+256.087586864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : secret "insights-runtime-extractor-tls" not found Sep 15 06:40:52.033023 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.032987 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d9de86b-d55c-49ef-83a8-677097aa9444-crio-socket\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.033262 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.033068 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d9de86b-d55c-49ef-83a8-677097aa9444-crio-socket\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.033262 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:52.033107 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:52.033262 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:52.033163 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:52.533148396 +0000 UTC m=+256.087805797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : configmap "kube-rbac-proxy" not found Sep 15 06:40:52.033262 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.033250 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d9de86b-d55c-49ef-83a8-677097aa9444-data-volume\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.042569 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.042552 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6m68\" (UniqueName: \"kubernetes.io/projected/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-api-access-p6m68\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.537583 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.537548 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.537749 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:52.537601 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:52.537749 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:52.537679 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:52.537749 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:52.537693 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:52.537749 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:52.537730 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:53.537717703 +0000 UTC m=+257.092375103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : configmap "kube-rbac-proxy" not found Sep 15 06:40:52.537749 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:52.537750 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:53.537735274 +0000 UTC m=+257.092392676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : secret "insights-runtime-extractor-tls" not found Sep 15 06:40:53.545776 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:53.545720 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:53.546183 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.545848 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:53.546183 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:53.545902 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:53.546183 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.545921 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:55.545905459 +0000 UTC m=+259.100562860 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : configmap "kube-rbac-proxy" not found Sep 15 06:40:53.546183 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.546008 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:53.546183 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.546063 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:55.54605187 +0000 UTC m=+259.100709274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : secret "insights-runtime-extractor-tls" not found Sep 15 06:40:53.725040 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:53.725014 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m48nb_27055683-09d8-4071-b297-5feb0eb4e3b8/dns-node-resolver/0.log" Sep 15 06:40:53.948488 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:53.948452 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:40:53.948647 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:53.948500 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:40:53.948647 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.948606 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 06:40:53.948647 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.948628 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfbb448f7-8d6tr: secret "image-registry-tls" not found Sep 15 06:40:53.948745 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.948679 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls podName:3cb1e519-f962-473d-837e-b4c98befa2ee nodeName:}" failed. No retries permitted until 2026-09-15 06:41:01.948666109 +0000 UTC m=+265.503323510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls") pod "image-registry-5cfbb448f7-8d6tr" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee") : secret "image-registry-tls" not found Sep 15 06:40:53.948745 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.948612 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:53.948745 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:53.948737 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls podName:82de95cc-b9c4-4546-b870-588edc3da907 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:01.948723958 +0000 UTC m=+265.503381361 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-2tg7m" (UID: "82de95cc-b9c4-4546-b870-588edc3da907") : secret "cluster-monitoring-operator-tls" not found Sep 15 06:40:54.725324 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:54.725296 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pjt59_a0e87755-014a-4616-b966-b66c76c86721/node-ca/0.log" Sep 15 06:40:55.525856 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:55.525828 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6dtk4_f8dda479-a879-433e-b4ca-b667b60c8c01/serve-healthcheck-canary/0.log" Sep 15 06:40:55.561118 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:55.561092 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:55.561221 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:55.561159 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:55.561280 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:55.561222 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:55.561280 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:55.561247 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:55.561375 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:55.561293 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:59.56127189 +0000 UTC m=+263.115929295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : configmap "kube-rbac-proxy" not found Sep 15 06:40:55.561375 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:55.561315 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:40:59.5613039 +0000 UTC m=+263.115961302 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : secret "insights-runtime-extractor-tls" not found Sep 15 06:40:59.589817 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:59.589713 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:59.589817 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:40:59.589781 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:40:59.590260 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:59.589850 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 15 06:40:59.590260 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:59.589853 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:40:59.590260 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:59.589913 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:07.589899237 +0000 UTC m=+271.144556637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : configmap "kube-rbac-proxy" not found Sep 15 06:40:59.590260 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:40:59.589934 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:07.589920737 +0000 UTC m=+271.144578138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : secret "insights-runtime-extractor-tls" not found Sep 15 06:41:02.006601 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:02.006556 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls\") pod \"image-registry-5cfbb448f7-8d6tr\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:41:02.006601 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:02.006608 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:41:02.007034 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:02.006706 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 15 06:41:02.007034 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:02.006730 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfbb448f7-8d6tr: secret "image-registry-tls" not found Sep 15 06:41:02.007034 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:02.006747 2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 15 06:41:02.007034 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:02.006798 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls podName:82de95cc-b9c4-4546-b870-588edc3da907 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:18.006781918 +0000 UTC m=+281.561439324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-2tg7m" (UID: "82de95cc-b9c4-4546-b870-588edc3da907") : secret "cluster-monitoring-operator-tls" not found Sep 15 06:41:02.007034 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:02.006812 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls podName:3cb1e519-f962-473d-837e-b4c98befa2ee nodeName:}" failed. No retries permitted until 2026-09-15 06:41:18.006805255 +0000 UTC m=+281.561462656 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls") pod "image-registry-5cfbb448f7-8d6tr" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee") : secret "image-registry-tls" not found Sep 15 06:41:04.396899 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:04.396852 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" podUID="a8c247d7-28c7-4fe8-8c34-f3ad56d96110" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Sep 15 06:41:04.693764 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:04.693736 2552 generic.go:356] "Generic (PLEG): container finished" podID="a8c247d7-28c7-4fe8-8c34-f3ad56d96110" containerID="d4d55eca92a2f6dd22ac3a03408e85b4e9db0531a0d8d835ad01935a070583d1" exitCode=1 Sep 15 06:41:04.693926 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:04.693773 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" event={"ID":"a8c247d7-28c7-4fe8-8c34-f3ad56d96110","Type":"ContainerDied","Data":"d4d55eca92a2f6dd22ac3a03408e85b4e9db0531a0d8d835ad01935a070583d1"} Sep 15 06:41:04.694066 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:04.694054 2552 scope.go:117] "RemoveContainer" containerID="d4d55eca92a2f6dd22ac3a03408e85b4e9db0531a0d8d835ad01935a070583d1" Sep 15 06:41:05.699518 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:05.699483 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" event={"ID":"a8c247d7-28c7-4fe8-8c34-f3ad56d96110","Type":"ContainerStarted","Data":"68f95ac7cbd9dfca24dc7b035f5909d399d9b90411b1ad4608e6056e58efb48c"} Sep 15 06:41:05.699888 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:05.699779 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:41:05.700389 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:05.700362 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c678594fd-ldc27" Sep 15 06:41:07.647951 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:07.647917 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:41:07.648434 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:07.648011 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:41:07.648434 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:07.648099 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 15 06:41:07.648434 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:07.648149 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls podName:3d9de86b-d55c-49ef-83a8-677097aa9444 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:23.648137 +0000 UTC m=+287.202794401 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls") pod "insights-runtime-extractor-84cfd" (UID: "3d9de86b-d55c-49ef-83a8-677097aa9444") : secret "insights-runtime-extractor-tls" not found Sep 15 06:41:07.648598 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:07.648536 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d9de86b-d55c-49ef-83a8-677097aa9444-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:41:08.710350 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:08.710316 2552 generic.go:356] "Generic (PLEG): container finished" podID="443aa338-2090-4ef7-8075-920e3cc8de4d" containerID="3c1eb1d4da1085547d191363b83e3aa99f89115a47623276085a69a836130df2" exitCode=255 Sep 15 06:41:08.710709 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:08.710363 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" event={"ID":"443aa338-2090-4ef7-8075-920e3cc8de4d","Type":"ContainerDied","Data":"3c1eb1d4da1085547d191363b83e3aa99f89115a47623276085a69a836130df2"} Sep 15 06:41:08.715833 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:08.715816 2552 scope.go:117] "RemoveContainer" containerID="3c1eb1d4da1085547d191363b83e3aa99f89115a47623276085a69a836130df2" Sep 15 06:41:09.649114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:09.649082 2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" Sep 15 06:41:09.721027 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:09.720985 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85d88bdf64-zkm7k" event={"ID":"443aa338-2090-4ef7-8075-920e3cc8de4d","Type":"ContainerStarted","Data":"90a0d81d51dce19b7e82caaa43d2cd1ce386b5bfed723c3a69a2ec73caf824bc"} Sep 15 06:41:15.533851 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.533816 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cfbb448f7-8d6tr"] Sep 15 06:41:15.534243 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:15.534022 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" podUID="3cb1e519-f962-473d-837e-b4c98befa2ee" Sep 15 06:41:15.567648 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.567620 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59bf595485-j25wn"] Sep 15 06:41:15.570701 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.570683 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.583305 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.583284 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59bf595485-j25wn"] Sep 15 06:41:15.599573 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.599552 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-bound-sa-token\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.599651 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.599584 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-ca-trust-extracted\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.599651 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.599601 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-registry-certificates\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.599651 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.599617 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz67m\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-kube-api-access-wz67m\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.599751 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.599724 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-image-registry-private-configuration\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.599784 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.599754 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-trusted-ca\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.599784 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.599772 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-registry-tls\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.599841 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.599790 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-installation-pull-secrets\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.700580 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.700554 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-registry-tls\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.700580 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.700583 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-installation-pull-secrets\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.700737 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.700643 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-bound-sa-token\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.700793 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.700758 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-ca-trust-extracted\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.700843 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.700801 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-registry-certificates\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.700843 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.700834 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz67m\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-kube-api-access-wz67m\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.701100 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.701073 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-image-registry-private-configuration\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.701184 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.701111 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-trusted-ca\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.701184 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.701168 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-ca-trust-extracted\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.701710 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.701687 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-registry-certificates\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.702279 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.702260 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-trusted-ca\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.703005 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.702987 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-installation-pull-secrets\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.703085 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.703074 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-registry-tls\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.703468 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.703439 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-image-registry-private-configuration\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.709602 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.709574 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-bound-sa-token\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.709821 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.709805 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz67m\" (UniqueName: \"kubernetes.io/projected/d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d-kube-api-access-wz67m\") pod \"image-registry-59bf595485-j25wn\" (UID: \"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d\") " pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.738633 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.738605 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:41:15.745783 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.745762 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:41:15.802002 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.801949 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-image-registry-private-configuration\") pod \"3cb1e519-f962-473d-837e-b4c98befa2ee\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " Sep 15 06:41:15.802002 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.801994 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsgg4\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-kube-api-access-bsgg4\") pod \"3cb1e519-f962-473d-837e-b4c98befa2ee\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " Sep 15 06:41:15.802120 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802030 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3cb1e519-f962-473d-837e-b4c98befa2ee-ca-trust-extracted\") pod \"3cb1e519-f962-473d-837e-b4c98befa2ee\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " Sep 15 06:41:15.802159 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802126 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-bound-sa-token\") pod \"3cb1e519-f962-473d-837e-b4c98befa2ee\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " Sep 15 06:41:15.802205 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802157 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-certificates\") pod \"3cb1e519-f962-473d-837e-b4c98befa2ee\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " Sep 15 06:41:15.802205 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802180 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-installation-pull-secrets\") pod \"3cb1e519-f962-473d-837e-b4c98befa2ee\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " Sep 15 06:41:15.802297 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802229 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-trusted-ca\") pod \"3cb1e519-f962-473d-837e-b4c98befa2ee\" (UID: \"3cb1e519-f962-473d-837e-b4c98befa2ee\") " Sep 15 06:41:15.802353 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802289 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cb1e519-f962-473d-837e-b4c98befa2ee-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3cb1e519-f962-473d-837e-b4c98befa2ee" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 06:41:15.802534 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802511 2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3cb1e519-f962-473d-837e-b4c98befa2ee-ca-trust-extracted\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:41:15.802627 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802547 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3cb1e519-f962-473d-837e-b4c98befa2ee" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:41:15.802697 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.802659 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3cb1e519-f962-473d-837e-b4c98befa2ee" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:41:15.804084 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.804059 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3cb1e519-f962-473d-837e-b4c98befa2ee" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:41:15.804376 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.804359 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3cb1e519-f962-473d-837e-b4c98befa2ee" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:41:15.804554 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.804534 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3cb1e519-f962-473d-837e-b4c98befa2ee" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:41:15.804618 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.804555 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-kube-api-access-bsgg4" (OuterVolumeSpecName: "kube-api-access-bsgg4") pod "3cb1e519-f962-473d-837e-b4c98befa2ee" (UID: "3cb1e519-f962-473d-837e-b4c98befa2ee"). InnerVolumeSpecName "kube-api-access-bsgg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:41:15.878564 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.878541 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:15.903853 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.903825 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsgg4\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-kube-api-access-bsgg4\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:41:15.903853 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.903852 2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-bound-sa-token\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:41:15.904111 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.903866 2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-certificates\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:41:15.904111 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.903878 2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-installation-pull-secrets\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:41:15.904111 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.903890 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3cb1e519-f962-473d-837e-b4c98befa2ee-trusted-ca\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:41:15.904111 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:15.903902 2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3cb1e519-f962-473d-837e-b4c98befa2ee-image-registry-private-configuration\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:41:16.145795 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.145666 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59bf595485-j25wn"] Sep 15 06:41:16.148173 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:16.148149 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9c8c85a_4fa4_43f7_a842_e8c3370dfa2d.slice/crio-a48b8275ba3504819d7501a2cb115216f23f5e883b0cf6bd8d75a549ad01d3d8 WatchSource:0}: Error finding container a48b8275ba3504819d7501a2cb115216f23f5e883b0cf6bd8d75a549ad01d3d8: Status 404 returned error can't find the container with id a48b8275ba3504819d7501a2cb115216f23f5e883b0cf6bd8d75a549ad01d3d8 Sep 15 06:41:16.423800 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:16.423709 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6nc7q" podUID="93f7caf1-70fc-43c2-b8f9-9710fbbf2cea" Sep 15 06:41:16.743538 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.743511 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6nc7q" Sep 15 06:41:16.744021 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.743538 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59bf595485-j25wn" event={"ID":"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d","Type":"ContainerStarted","Data":"6e29b329eccd7748d8b7f03b3903949d730cb658062e22c14386064f41dece9d"} Sep 15 06:41:16.744021 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.743609 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59bf595485-j25wn" event={"ID":"d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d","Type":"ContainerStarted","Data":"a48b8275ba3504819d7501a2cb115216f23f5e883b0cf6bd8d75a549ad01d3d8"} Sep 15 06:41:16.744021 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.743660 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:16.744021 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.743669 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfbb448f7-8d6tr" Sep 15 06:41:16.762827 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.762785 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59bf595485-j25wn" podStartSLOduration=1.762760824 podStartE2EDuration="1.762760824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:16.76140013 +0000 UTC m=+280.316057550" watchObservedRunningTime="2026-09-15 06:41:16.762760824 +0000 UTC m=+280.317418247" Sep 15 06:41:16.785147 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.785122 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cfbb448f7-8d6tr"] Sep 15 06:41:16.788168 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.788147 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cfbb448f7-8d6tr"] Sep 15 06:41:16.810472 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.810362 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3cb1e519-f962-473d-837e-b4c98befa2ee-registry-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:41:16.859843 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:16.859810 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cb1e519-f962-473d-837e-b4c98befa2ee" path="/var/lib/kubelet/pods/3cb1e519-f962-473d-837e-b4c98befa2ee/volumes" Sep 15 06:41:18.020052 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:18.020011 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:41:18.022319 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:18.022299 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/82de95cc-b9c4-4546-b870-588edc3da907-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-2tg7m\" (UID: \"82de95cc-b9c4-4546-b870-588edc3da907\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:41:18.320666 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:18.320591 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" Sep 15 06:41:18.588186 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:18.588163 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m"] Sep 15 06:41:18.590394 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:18.590364 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82de95cc_b9c4_4546_b870_588edc3da907.slice/crio-4f503f510fa97836b57d9499b1e24f7ca6ac029960b3e0799aec7b5963517bc0 WatchSource:0}: Error finding container 4f503f510fa97836b57d9499b1e24f7ca6ac029960b3e0799aec7b5963517bc0: Status 404 returned error can't find the container with id 4f503f510fa97836b57d9499b1e24f7ca6ac029960b3e0799aec7b5963517bc0 Sep 15 06:41:18.750891 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:18.750859 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" event={"ID":"82de95cc-b9c4-4546-b870-588edc3da907","Type":"ContainerStarted","Data":"4f503f510fa97836b57d9499b1e24f7ca6ac029960b3e0799aec7b5963517bc0"} Sep 15 06:41:20.238901 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:20.238868 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:41:20.241114 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:20.241089 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/93f7caf1-70fc-43c2-b8f9-9710fbbf2cea-metrics-tls\") pod \"dns-default-6nc7q\" (UID: \"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea\") " pod="openshift-dns/dns-default-6nc7q" Sep 15 06:41:20.346667 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:20.346599 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k55p\"" Sep 15 06:41:20.354839 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:20.354816 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6nc7q" Sep 15 06:41:20.624111 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:20.624088 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6nc7q"] Sep 15 06:41:20.626559 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:20.626534 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93f7caf1_70fc_43c2_b8f9_9710fbbf2cea.slice/crio-a6f911a5c6a415fde9979b09421e624374c47169b1935e91c58c62e6b22c7916 WatchSource:0}: Error finding container a6f911a5c6a415fde9979b09421e624374c47169b1935e91c58c62e6b22c7916: Status 404 returned error can't find the container with id a6f911a5c6a415fde9979b09421e624374c47169b1935e91c58c62e6b22c7916 Sep 15 06:41:20.759486 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:20.759449 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" event={"ID":"82de95cc-b9c4-4546-b870-588edc3da907","Type":"ContainerStarted","Data":"920ab19a0db43f909d85d5d4e4ae8e7fe1ffefa4b9f0fe491b889bd369683a37"} Sep 15 06:41:20.760612 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:20.760586 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6nc7q" event={"ID":"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea","Type":"ContainerStarted","Data":"a6f911a5c6a415fde9979b09421e624374c47169b1935e91c58c62e6b22c7916"} Sep 15 06:41:20.776652 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:20.776612 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-2tg7m" podStartSLOduration=33.279410365 podStartE2EDuration="34.776597911s" totalImagesPullingTime="1.497187546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:20.776059737 +0000 UTC m=+284.330717162" watchObservedRunningTime="2026-09-15 06:41:20.776597911 +0000 UTC m=+284.331255333" Sep 15 06:41:23.630111 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.630074 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5"] Sep 15 06:41:23.633720 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.633692 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.637707 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.637629 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 15 06:41:23.637707 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.637645 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 15 06:41:23.637707 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.637656 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rn945\"" Sep 15 06:41:23.638000 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.637724 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 15 06:41:23.642091 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.641604 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5"] Sep 15 06:41:23.664636 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.664594 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:41:23.664831 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.664659 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c489087f-e151-472a-a419-6a1791d8b5d4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.664831 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.664681 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c489087f-e151-472a-a419-6a1791d8b5d4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.664831 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.664714 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95zxm\" (UniqueName: \"kubernetes.io/projected/c489087f-e151-472a-a419-6a1791d8b5d4-kube-api-access-95zxm\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.664831 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.664756 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c489087f-e151-472a-a419-6a1791d8b5d4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.667251 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.667229 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d9de86b-d55c-49ef-83a8-677097aa9444-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-84cfd\" (UID: \"3d9de86b-d55c-49ef-83a8-677097aa9444\") " pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:41:23.765378 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.765341 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c489087f-e151-472a-a419-6a1791d8b5d4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.765542 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.765404 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c489087f-e151-472a-a419-6a1791d8b5d4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.765542 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.765432 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c489087f-e151-472a-a419-6a1791d8b5d4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.765542 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.765459 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95zxm\" (UniqueName: \"kubernetes.io/projected/c489087f-e151-472a-a419-6a1791d8b5d4-kube-api-access-95zxm\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.766088 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.766058 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c489087f-e151-472a-a419-6a1791d8b5d4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.767908 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.767886 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c489087f-e151-472a-a419-6a1791d8b5d4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.768031 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.767994 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c489087f-e151-472a-a419-6a1791d8b5d4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.773927 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.773905 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95zxm\" (UniqueName: \"kubernetes.io/projected/c489087f-e151-472a-a419-6a1791d8b5d4-kube-api-access-95zxm\") pod \"prometheus-operator-68bcd7f987-hx6k5\" (UID: \"c489087f-e151-472a-a419-6a1791d8b5d4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:23.869262 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.869241 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-84cfd" Sep 15 06:41:23.945069 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:23.945037 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" Sep 15 06:41:24.149957 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.149880 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-84cfd"] Sep 15 06:41:24.153770 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:24.153741 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9de86b_d55c_49ef_83a8_677097aa9444.slice/crio-fcc97ab95ec0aa01c72d51cb07d23aadf2cb9a8af07c320eb1dcdaf697b58351 WatchSource:0}: Error finding container fcc97ab95ec0aa01c72d51cb07d23aadf2cb9a8af07c320eb1dcdaf697b58351: Status 404 returned error can't find the container with id fcc97ab95ec0aa01c72d51cb07d23aadf2cb9a8af07c320eb1dcdaf697b58351 Sep 15 06:41:24.218202 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.218177 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5"] Sep 15 06:41:24.220918 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:24.220884 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc489087f_e151_472a_a419_6a1791d8b5d4.slice/crio-387a5aeaeef6470f0a86fbfde8472e0ae717745f11442d9e5937312d6b68e6a9 WatchSource:0}: Error finding container 387a5aeaeef6470f0a86fbfde8472e0ae717745f11442d9e5937312d6b68e6a9: Status 404 returned error can't find the container with id 387a5aeaeef6470f0a86fbfde8472e0ae717745f11442d9e5937312d6b68e6a9 Sep 15 06:41:24.774459 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.774427 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" event={"ID":"c489087f-e151-472a-a419-6a1791d8b5d4","Type":"ContainerStarted","Data":"387a5aeaeef6470f0a86fbfde8472e0ae717745f11442d9e5937312d6b68e6a9"} Sep 15 06:41:24.775897 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.775871 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84cfd" event={"ID":"3d9de86b-d55c-49ef-83a8-677097aa9444","Type":"ContainerStarted","Data":"34595ab0498caa68ae74c4a44c8465dfebf03de01090a56eb95d918e8905bbc1"} Sep 15 06:41:24.776044 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.775924 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84cfd" event={"ID":"3d9de86b-d55c-49ef-83a8-677097aa9444","Type":"ContainerStarted","Data":"fcc97ab95ec0aa01c72d51cb07d23aadf2cb9a8af07c320eb1dcdaf697b58351"} Sep 15 06:41:24.777464 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.777440 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6nc7q" event={"ID":"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea","Type":"ContainerStarted","Data":"c52abc82479dbd13b75c578b859e3d62cf847bb55383b722f5be3436e40649ea"} Sep 15 06:41:24.777570 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.777487 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6nc7q" event={"ID":"93f7caf1-70fc-43c2-b8f9-9710fbbf2cea","Type":"ContainerStarted","Data":"043992c8828d96bada1affd70c91d9a61093dee8150659a656dbc6b677e79349"} Sep 15 06:41:24.777621 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.777593 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6nc7q" Sep 15 06:41:24.793742 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:24.793678 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6nc7q" podStartSLOduration=251.629225898 podStartE2EDuration="4m14.793658033s" totalImagesPullingTime="3.164432135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:24.793407936 +0000 UTC m=+288.348065371" watchObservedRunningTime="2026-09-15 06:41:24.793658033 +0000 UTC m=+288.348315454" Sep 15 06:41:25.783281 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:25.783240 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84cfd" event={"ID":"3d9de86b-d55c-49ef-83a8-677097aa9444","Type":"ContainerStarted","Data":"d6efbde1ddd19a81d685586df1849d7a990ae400abe2bf58b625f315f618cfed"} Sep 15 06:41:26.789554 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:26.789517 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-84cfd" event={"ID":"3d9de86b-d55c-49ef-83a8-677097aa9444","Type":"ContainerStarted","Data":"161c01b60b6b080819fa2025f1e4bda462b04efbb6eac23ad0c2da5a93204474"} Sep 15 06:41:26.807374 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:26.807313 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-84cfd" podStartSLOduration=33.47658878 podStartE2EDuration="35.807295369s" totalImagesPullingTime="2.330706589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 06:40:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:26.806244731 +0000 UTC m=+290.360902154" watchObservedRunningTime="2026-09-15 06:41:26.807295369 +0000 UTC m=+290.361952774" Sep 15 06:41:27.794634 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:27.794538 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" event={"ID":"c489087f-e151-472a-a419-6a1791d8b5d4","Type":"ContainerStarted","Data":"5fddacd72e2c2e221256765374206a2b8bc93eb88707ae7c32f8709992430f85"} Sep 15 06:41:27.794634 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:27.794594 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" event={"ID":"c489087f-e151-472a-a419-6a1791d8b5d4","Type":"ContainerStarted","Data":"7b1551439d7c4faf3f0fe846795d2b5f961494a75d1e94c1ae73151b50e37eeb"} Sep 15 06:41:27.810619 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:27.810573 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-hx6k5" podStartSLOduration=1.519406024 podStartE2EDuration="4.810556053s" totalImagesPullingTime="3.291150029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:27.810009519 +0000 UTC m=+291.364666943" watchObservedRunningTime="2026-09-15 06:41:27.810556053 +0000 UTC m=+291.365213498" Sep 15 06:41:28.905193 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:28.905165 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-gw6xp"] Sep 15 06:41:28.908012 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:28.907994 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-gw6xp" Sep 15 06:41:28.911563 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:28.910821 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-nbc6r\"" Sep 15 06:41:28.912846 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:28.912820 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 15 06:41:28.913172 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:28.913125 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 15 06:41:28.916408 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:28.916387 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-gw6xp"] Sep 15 06:41:29.005003 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.004946 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6762\" (UniqueName: \"kubernetes.io/projected/bad6582f-6851-4bae-91f6-ad033be4414d-kube-api-access-k6762\") pod \"downloads-76f99584f4-gw6xp\" (UID: \"bad6582f-6851-4bae-91f6-ad033be4414d\") " pod="openshift-console/downloads-76f99584f4-gw6xp" Sep 15 06:41:29.105432 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.105396 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6762\" (UniqueName: \"kubernetes.io/projected/bad6582f-6851-4bae-91f6-ad033be4414d-kube-api-access-k6762\") pod \"downloads-76f99584f4-gw6xp\" (UID: \"bad6582f-6851-4bae-91f6-ad033be4414d\") " pod="openshift-console/downloads-76f99584f4-gw6xp" Sep 15 06:41:29.113040 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.113006 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6762\" (UniqueName: \"kubernetes.io/projected/bad6582f-6851-4bae-91f6-ad033be4414d-kube-api-access-k6762\") pod \"downloads-76f99584f4-gw6xp\" (UID: \"bad6582f-6851-4bae-91f6-ad033be4414d\") " pod="openshift-console/downloads-76f99584f4-gw6xp" Sep 15 06:41:29.219314 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.219284 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-gw6xp" Sep 15 06:41:29.484443 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.484369 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-gw6xp"] Sep 15 06:41:29.487449 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:29.487422 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad6582f_6851_4bae_91f6_ad033be4414d.slice/crio-7d1e5a5ad33d7677c383aa6749a7e0b5e4cd05093b12f07deb670147f6c93b45 WatchSource:0}: Error finding container 7d1e5a5ad33d7677c383aa6749a7e0b5e4cd05093b12f07deb670147f6c93b45: Status 404 returned error can't find the container with id 7d1e5a5ad33d7677c383aa6749a7e0b5e4cd05093b12f07deb670147f6c93b45 Sep 15 06:41:29.802236 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.802160 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-gw6xp" event={"ID":"bad6582f-6851-4bae-91f6-ad033be4414d","Type":"ContainerStarted","Data":"7d1e5a5ad33d7677c383aa6749a7e0b5e4cd05093b12f07deb670147f6c93b45"} Sep 15 06:41:29.977137 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.977104 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd"] Sep 15 06:41:29.980562 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.980539 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:29.983137 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.983103 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 15 06:41:29.983232 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.983111 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wp6nx\"" Sep 15 06:41:29.983232 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.983147 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 15 06:41:29.983232 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.983140 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 15 06:41:29.985716 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.985697 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-622vl"] Sep 15 06:41:29.988773 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.988753 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:29.990991 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.990959 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 15 06:41:29.991556 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.991539 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 15 06:41:29.991790 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.991773 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd"] Sep 15 06:41:29.991981 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.991943 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2dkmn\"" Sep 15 06:41:29.992069 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:29.992053 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 15 06:41:30.010758 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.010733 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-textfile\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.010846 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.010767 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-root\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.010846 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.010791 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af51d317-972e-49c1-8e27-a77632a4fa97-metrics-client-ca\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.010938 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.010851 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7ca278c-776f-45ce-abc1-9416bb615cd0-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.010938 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.010881 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8z5j\" (UniqueName: \"kubernetes.io/projected/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-api-access-n8z5j\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.010938 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.010910 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-accelerators-collector-config\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.011116 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.010984 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a7ca278c-776f-45ce-abc1-9416bb615cd0-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.011116 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.011027 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.011116 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.011075 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.011243 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.011123 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q8cf\" (UniqueName: \"kubernetes.io/projected/af51d317-972e-49c1-8e27-a77632a4fa97-kube-api-access-8q8cf\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.011243 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.011149 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.011243 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.011202 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.011378 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.011243 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-tls\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.011378 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.011286 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-sys\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.011378 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.011305 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-wtmp\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.111847 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.111756 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7ca278c-776f-45ce-abc1-9416bb615cd0-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.111847 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.111808 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8z5j\" (UniqueName: \"kubernetes.io/projected/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-api-access-n8z5j\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.112085 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.111947 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-accelerators-collector-config\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112085 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112011 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a7ca278c-776f-45ce-abc1-9416bb615cd0-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.112085 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112046 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112247 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112143 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.112247 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112188 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q8cf\" (UniqueName: \"kubernetes.io/projected/af51d317-972e-49c1-8e27-a77632a4fa97-kube-api-access-8q8cf\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112247 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112214 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.112390 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:30.112314 2552 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 15 06:41:30.112390 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:30.112370 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-tls podName:a7ca278c-776f-45ce-abc1-9416bb615cd0 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:30.612352758 +0000 UTC m=+294.167010167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-tz6fd" (UID: "a7ca278c-776f-45ce-abc1-9416bb615cd0") : secret "kube-state-metrics-tls" not found Sep 15 06:41:30.112497 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112474 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.112553 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112515 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-tls\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112553 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112527 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a7ca278c-776f-45ce-abc1-9416bb615cd0-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.112669 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112574 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-sys\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112669 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112601 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-wtmp\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112669 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112610 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7ca278c-776f-45ce-abc1-9416bb615cd0-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.112669 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112633 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-textfile\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112669 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:30.112615 2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 15 06:41:30.112905 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:30.112695 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-tls podName:af51d317-972e-49c1-8e27-a77632a4fa97 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:30.612679542 +0000 UTC m=+294.167336948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-tls") pod "node-exporter-622vl" (UID: "af51d317-972e-49c1-8e27-a77632a4fa97") : secret "node-exporter-tls" not found Sep 15 06:41:30.112905 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112757 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-wtmp\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112905 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112791 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-root\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112905 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112816 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af51d317-972e-49c1-8e27-a77632a4fa97-metrics-client-ca\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.112905 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112848 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-accelerators-collector-config\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.113175 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112908 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-sys\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.113175 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112911 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af51d317-972e-49c1-8e27-a77632a4fa97-root\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.113175 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.112958 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-textfile\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.113325 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.113180 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.113431 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.113381 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af51d317-972e-49c1-8e27-a77632a4fa97-metrics-client-ca\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.115181 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.115158 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.115276 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.115231 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.125376 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.125353 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8z5j\" (UniqueName: \"kubernetes.io/projected/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-api-access-n8z5j\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.125481 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.125439 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q8cf\" (UniqueName: \"kubernetes.io/projected/af51d317-972e-49c1-8e27-a77632a4fa97-kube-api-access-8q8cf\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.617584 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.617541 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-tls\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.617775 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.617666 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.620164 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.620138 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7ca278c-776f-45ce-abc1-9416bb615cd0-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-tz6fd\" (UID: \"a7ca278c-776f-45ce-abc1-9416bb615cd0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.620164 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.620155 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af51d317-972e-49c1-8e27-a77632a4fa97-node-exporter-tls\") pod \"node-exporter-622vl\" (UID: \"af51d317-972e-49c1-8e27-a77632a4fa97\") " pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.889787 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.889713 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" Sep 15 06:41:30.897191 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:30.897169 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-622vl" Sep 15 06:41:30.910303 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:30.909707 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf51d317_972e_49c1_8e27_a77632a4fa97.slice/crio-f6c9b1d7b7bddbfad54481b068420aff86b9d5640a0444000cb3a268c9696a75 WatchSource:0}: Error finding container f6c9b1d7b7bddbfad54481b068420aff86b9d5640a0444000cb3a268c9696a75: Status 404 returned error can't find the container with id f6c9b1d7b7bddbfad54481b068420aff86b9d5640a0444000cb3a268c9696a75 Sep 15 06:41:31.185629 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:31.185593 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd"] Sep 15 06:41:31.189255 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:31.189227 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ca278c_776f_45ce_abc1_9416bb615cd0.slice/crio-af65e3b53e690f9e52bc1fd1f7786610f80cd27debc0e70620064506fbd772cb WatchSource:0}: Error finding container af65e3b53e690f9e52bc1fd1f7786610f80cd27debc0e70620064506fbd772cb: Status 404 returned error can't find the container with id af65e3b53e690f9e52bc1fd1f7786610f80cd27debc0e70620064506fbd772cb Sep 15 06:41:31.814090 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:31.814047 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" event={"ID":"a7ca278c-776f-45ce-abc1-9416bb615cd0","Type":"ContainerStarted","Data":"af65e3b53e690f9e52bc1fd1f7786610f80cd27debc0e70620064506fbd772cb"} Sep 15 06:41:31.818002 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:31.816594 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-622vl" event={"ID":"af51d317-972e-49c1-8e27-a77632a4fa97","Type":"ContainerStarted","Data":"f6c9b1d7b7bddbfad54481b068420aff86b9d5640a0444000cb3a268c9696a75"} Sep 15 06:41:31.856285 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:31.856243 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ns6cw" podUID="532a12f1-78ac-4e4d-a02d-6be6c808825f" Sep 15 06:41:32.829077 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:32.828794 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" event={"ID":"a7ca278c-776f-45ce-abc1-9416bb615cd0","Type":"ContainerStarted","Data":"3f63e1d3f0f213b386959f9a77190230f3660562e2da28db7d4a02c1f92d92d9"} Sep 15 06:41:32.829077 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:32.828901 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" event={"ID":"a7ca278c-776f-45ce-abc1-9416bb615cd0","Type":"ContainerStarted","Data":"1a1a436c3ac5032ba4f7ecbe9e159438faf47f83b699c0e1876e65f28c385b89"} Sep 15 06:41:32.833017 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:32.831584 2552 generic.go:356] "Generic (PLEG): container finished" podID="af51d317-972e-49c1-8e27-a77632a4fa97" containerID="f686881906b7bfdcbd892cb5a6d335eaa9ea08a544e7b54e25f469988a92a804" exitCode=0 Sep 15 06:41:32.833017 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:32.831642 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-622vl" event={"ID":"af51d317-972e-49c1-8e27-a77632a4fa97","Type":"ContainerDied","Data":"f686881906b7bfdcbd892cb5a6d335eaa9ea08a544e7b54e25f469988a92a804"} Sep 15 06:41:33.838478 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:33.838438 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" event={"ID":"a7ca278c-776f-45ce-abc1-9416bb615cd0","Type":"ContainerStarted","Data":"10171193fd77d75615d277c8455152c27eb35eccaf81cbb8d45bfbca283e36ae"} Sep 15 06:41:33.840931 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:33.840900 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-622vl" event={"ID":"af51d317-972e-49c1-8e27-a77632a4fa97","Type":"ContainerStarted","Data":"e3f064c86fb7e53a9485133543d822e73dd03b03bc2bff5f118c017a3369c253"} Sep 15 06:41:33.841073 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:33.840981 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-622vl" event={"ID":"af51d317-972e-49c1-8e27-a77632a4fa97","Type":"ContainerStarted","Data":"fbb9ba90f7c1133e6688bb1bdb82e9284a1a71c33b03d90807c3931524931d4c"} Sep 15 06:41:33.857267 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:33.857224 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-tz6fd" podStartSLOduration=3.397742151 podStartE2EDuration="4.857206878s" totalImagesPullingTime="1.459464727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:33.855552771 +0000 UTC m=+297.410210197" watchObservedRunningTime="2026-09-15 06:41:33.857206878 +0000 UTC m=+297.411864301" Sep 15 06:41:33.872610 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:33.872566 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-622vl" podStartSLOduration=3.635944634 podStartE2EDuration="4.872547969s" totalImagesPullingTime="1.158243224s" totalInitContainerRuntime="78.360111ms" isStatefulPod=true podCreationTimestamp="2026-09-15 06:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:33.87099062 +0000 UTC m=+297.425648046" watchObservedRunningTime="2026-09-15 06:41:33.872547969 +0000 UTC m=+297.427205387" Sep 15 06:41:34.740262 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.740231 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5"] Sep 15 06:41:34.743392 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.743373 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:34.745988 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.745940 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dj5nq\"" Sep 15 06:41:34.746093 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.745993 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 15 06:41:34.750076 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.750052 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5"] Sep 15 06:41:34.785618 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.785592 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6nc7q" Sep 15 06:41:34.858793 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.858765 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsbqn\" (UniqueName: \"kubernetes.io/projected/75cc5b37-f8e9-481f-a034-16729e6ceb07-kube-api-access-wsbqn\") pod \"monitoring-plugin-5d85778b9d-rhbx5\" (UID: \"75cc5b37-f8e9-481f-a034-16729e6ceb07\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:34.859240 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.858911 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75cc5b37-f8e9-481f-a034-16729e6ceb07-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-rhbx5\" (UID: \"75cc5b37-f8e9-481f-a034-16729e6ceb07\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:34.959817 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.959776 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsbqn\" (UniqueName: \"kubernetes.io/projected/75cc5b37-f8e9-481f-a034-16729e6ceb07-kube-api-access-wsbqn\") pod \"monitoring-plugin-5d85778b9d-rhbx5\" (UID: \"75cc5b37-f8e9-481f-a034-16729e6ceb07\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:34.960438 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.960126 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75cc5b37-f8e9-481f-a034-16729e6ceb07-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-rhbx5\" (UID: \"75cc5b37-f8e9-481f-a034-16729e6ceb07\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:34.960438 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:34.960204 2552 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 15 06:41:34.960438 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:41:34.960259 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75cc5b37-f8e9-481f-a034-16729e6ceb07-monitoring-plugin-cert podName:75cc5b37-f8e9-481f-a034-16729e6ceb07 nodeName:}" failed. No retries permitted until 2026-09-15 06:41:35.460239675 +0000 UTC m=+299.014897080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/75cc5b37-f8e9-481f-a034-16729e6ceb07-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-rhbx5" (UID: "75cc5b37-f8e9-481f-a034-16729e6ceb07") : secret "monitoring-plugin-cert" not found Sep 15 06:41:34.968394 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:34.968366 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsbqn\" (UniqueName: \"kubernetes.io/projected/75cc5b37-f8e9-481f-a034-16729e6ceb07-kube-api-access-wsbqn\") pod \"monitoring-plugin-5d85778b9d-rhbx5\" (UID: \"75cc5b37-f8e9-481f-a034-16729e6ceb07\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:35.464259 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:35.464225 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75cc5b37-f8e9-481f-a034-16729e6ceb07-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-rhbx5\" (UID: \"75cc5b37-f8e9-481f-a034-16729e6ceb07\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:35.466738 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:35.466710 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/75cc5b37-f8e9-481f-a034-16729e6ceb07-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-rhbx5\" (UID: \"75cc5b37-f8e9-481f-a034-16729e6ceb07\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:35.656663 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:35.656629 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:35.934756 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:35.934692 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5"] Sep 15 06:41:35.937799 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:35.937770 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75cc5b37_f8e9_481f_a034_16729e6ceb07.slice/crio-43810f4160ee1161083fb6019239b7ce39a5ce9282ae8941b7684d186d52e43a WatchSource:0}: Error finding container 43810f4160ee1161083fb6019239b7ce39a5ce9282ae8941b7684d186d52e43a: Status 404 returned error can't find the container with id 43810f4160ee1161083fb6019239b7ce39a5ce9282ae8941b7684d186d52e43a Sep 15 06:41:36.798594 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:36.798567 2552 kubelet.go:1648] "Image garbage collection succeeded" Sep 15 06:41:36.861948 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:36.861808 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" event={"ID":"75cc5b37-f8e9-481f-a034-16729e6ceb07","Type":"ContainerStarted","Data":"43810f4160ee1161083fb6019239b7ce39a5ce9282ae8941b7684d186d52e43a"} Sep 15 06:41:37.751532 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:37.751457 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59bf595485-j25wn" Sep 15 06:41:37.865559 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:37.865520 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" event={"ID":"75cc5b37-f8e9-481f-a034-16729e6ceb07","Type":"ContainerStarted","Data":"a0020adeeb0201c498a22cefe6f4946d9237ea39ce80597b4a8ae42e6877df7e"} Sep 15 06:41:37.865737 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:37.865709 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:37.871153 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:37.871131 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" Sep 15 06:41:37.880907 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:37.880865 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rhbx5" podStartSLOduration=2.330028257 podStartE2EDuration="3.880849127s" totalImagesPullingTime="1.55082087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:37.880138605 +0000 UTC m=+301.434796049" watchObservedRunningTime="2026-09-15 06:41:37.880849127 +0000 UTC m=+301.435506550" Sep 15 06:41:39.914300 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.914267 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-664869df9f-v57vl"] Sep 15 06:41:39.918575 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.918552 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:39.921233 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.921204 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 15 06:41:39.922503 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.922483 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 15 06:41:39.922597 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.922499 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 15 06:41:39.922597 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.922550 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ksjg5\"" Sep 15 06:41:39.922597 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.922501 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 15 06:41:39.922597 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.922556 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 15 06:41:39.925622 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:39.925602 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664869df9f-v57vl"] Sep 15 06:41:40.011155 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.011118 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-serving-cert\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.011324 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.011158 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-oauth-config\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.011324 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.011212 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-oauth-serving-cert\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.011324 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.011261 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsz64\" (UniqueName: \"kubernetes.io/projected/975fc5fb-7c9c-4bff-ad02-71bc52edd068-kube-api-access-zsz64\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.011466 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.011339 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-config\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.011466 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.011431 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-service-ca\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.112187 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.112146 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-oauth-serving-cert\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.112373 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.112200 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsz64\" (UniqueName: \"kubernetes.io/projected/975fc5fb-7c9c-4bff-ad02-71bc52edd068-kube-api-access-zsz64\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.112373 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.112230 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-config\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.112373 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.112288 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-service-ca\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.112373 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.112323 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-serving-cert\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.112373 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.112341 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-oauth-config\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.113035 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.113009 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-oauth-serving-cert\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.113275 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.113231 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-config\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.113275 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.113231 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-service-ca\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.115027 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.115008 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-oauth-config\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.115197 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.115182 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-serving-cert\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.120782 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.120761 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsz64\" (UniqueName: \"kubernetes.io/projected/975fc5fb-7c9c-4bff-ad02-71bc52edd068-kube-api-access-zsz64\") pod \"console-664869df9f-v57vl\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:40.229685 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:40.229645 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:41:44.855900 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:44.855861 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:41:46.527570 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:46.527461 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664869df9f-v57vl"] Sep 15 06:41:46.530454 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:41:46.530421 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975fc5fb_7c9c_4bff_ad02_71bc52edd068.slice/crio-4f4c691045afd8d5d92d9d38a3fa3939d6744db9eda0bbd87d3667ac362c9a17 WatchSource:0}: Error finding container 4f4c691045afd8d5d92d9d38a3fa3939d6744db9eda0bbd87d3667ac362c9a17: Status 404 returned error can't find the container with id 4f4c691045afd8d5d92d9d38a3fa3939d6744db9eda0bbd87d3667ac362c9a17 Sep 15 06:41:46.532919 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:46.532899 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 06:41:46.900626 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:46.900543 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664869df9f-v57vl" event={"ID":"975fc5fb-7c9c-4bff-ad02-71bc52edd068","Type":"ContainerStarted","Data":"4f4c691045afd8d5d92d9d38a3fa3939d6744db9eda0bbd87d3667ac362c9a17"} Sep 15 06:41:46.902370 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:46.902337 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-gw6xp" event={"ID":"bad6582f-6851-4bae-91f6-ad033be4414d","Type":"ContainerStarted","Data":"b67edbd758e64dbd41261e4bd61859a49bde505ea585f3135ae8f7427ce42fa8"} Sep 15 06:41:46.902578 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:46.902556 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-gw6xp" Sep 15 06:41:46.912618 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:46.912591 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-gw6xp" Sep 15 06:41:46.926248 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:46.926194 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-gw6xp" podStartSLOduration=2.06430483 podStartE2EDuration="18.926167414s" totalImagesPullingTime="16.861862584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:46.924868435 +0000 UTC m=+310.479525857" watchObservedRunningTime="2026-09-15 06:41:46.926167414 +0000 UTC m=+310.480824837" Sep 15 06:41:50.923665 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:50.923581 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664869df9f-v57vl" event={"ID":"975fc5fb-7c9c-4bff-ad02-71bc52edd068","Type":"ContainerStarted","Data":"5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a"} Sep 15 06:41:50.939747 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:50.939666 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-664869df9f-v57vl" podStartSLOduration=8.015662967 podStartE2EDuration="11.939651088s" totalImagesPullingTime="3.923988121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:41:50.938871996 +0000 UTC m=+314.493529420" watchObservedRunningTime="2026-09-15 06:41:50.939651088 +0000 UTC m=+314.494308511" Sep 15 06:41:59.670524 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:41:59.670489 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664869df9f-v57vl"] Sep 15 06:42:00.230749 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:00.230716 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:42:17.524227 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:17.524197 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6dtk4_f8dda479-a879-433e-b4ca-b667b60c8c01/serve-healthcheck-canary/0.log" Sep 15 06:42:24.688815 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:24.688773 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-664869df9f-v57vl" podUID="975fc5fb-7c9c-4bff-ad02-71bc52edd068" containerName="console" containerID="cri-o://5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a" gracePeriod=15 Sep 15 06:42:24.990297 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:24.990270 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:42:25.044019 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.043959 2552 generic.go:356] "Generic (PLEG): container finished" podID="975fc5fb-7c9c-4bff-ad02-71bc52edd068" containerID="5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a" exitCode=0 Sep 15 06:42:25.044019 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.044020 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664869df9f-v57vl" event={"ID":"975fc5fb-7c9c-4bff-ad02-71bc52edd068","Type":"ContainerDied","Data":"5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a"} Sep 15 06:42:25.044019 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.044023 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664869df9f-v57vl" Sep 15 06:42:25.044292 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.044060 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664869df9f-v57vl" event={"ID":"975fc5fb-7c9c-4bff-ad02-71bc52edd068","Type":"ContainerDied","Data":"4f4c691045afd8d5d92d9d38a3fa3939d6744db9eda0bbd87d3667ac362c9a17"} Sep 15 06:42:25.044292 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.044079 2552 scope.go:117] "RemoveContainer" containerID="5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a" Sep 15 06:42:25.066063 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.066041 2552 scope.go:117] "RemoveContainer" containerID="5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a" Sep 15 06:42:25.066367 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:42:25.066347 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a\": container with ID starting with 5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a not found: ID does not exist" containerID="5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a" Sep 15 06:42:25.066438 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.066381 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a"} err="failed to get container status \"5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a\": rpc error: code = NotFound desc = could not find container \"5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a\": container with ID starting with 5066a233f89752c0e7c8571d828afccb571a868bbb786bef799d4123a673c60a not found: ID does not exist" Sep 15 06:42:25.088616 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.088593 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-serving-cert\") pod \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " Sep 15 06:42:25.088710 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.088638 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsz64\" (UniqueName: \"kubernetes.io/projected/975fc5fb-7c9c-4bff-ad02-71bc52edd068-kube-api-access-zsz64\") pod \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " Sep 15 06:42:25.088710 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.088658 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-oauth-serving-cert\") pod \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " Sep 15 06:42:25.088710 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.088678 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-oauth-config\") pod \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " Sep 15 06:42:25.088837 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.088817 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-service-ca\") pod \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " Sep 15 06:42:25.088924 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.088907 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-config\") pod \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\" (UID: \"975fc5fb-7c9c-4bff-ad02-71bc52edd068\") " Sep 15 06:42:25.089163 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.089133 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "975fc5fb-7c9c-4bff-ad02-71bc52edd068" (UID: "975fc5fb-7c9c-4bff-ad02-71bc52edd068"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:42:25.089261 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.089217 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-service-ca" (OuterVolumeSpecName: "service-ca") pod "975fc5fb-7c9c-4bff-ad02-71bc52edd068" (UID: "975fc5fb-7c9c-4bff-ad02-71bc52edd068"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:42:25.089261 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.089243 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-config" (OuterVolumeSpecName: "console-config") pod "975fc5fb-7c9c-4bff-ad02-71bc52edd068" (UID: "975fc5fb-7c9c-4bff-ad02-71bc52edd068"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:42:25.090892 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.090863 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "975fc5fb-7c9c-4bff-ad02-71bc52edd068" (UID: "975fc5fb-7c9c-4bff-ad02-71bc52edd068"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:25.090993 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.090919 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "975fc5fb-7c9c-4bff-ad02-71bc52edd068" (UID: "975fc5fb-7c9c-4bff-ad02-71bc52edd068"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:42:25.090993 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.090921 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975fc5fb-7c9c-4bff-ad02-71bc52edd068-kube-api-access-zsz64" (OuterVolumeSpecName: "kube-api-access-zsz64") pod "975fc5fb-7c9c-4bff-ad02-71bc52edd068" (UID: "975fc5fb-7c9c-4bff-ad02-71bc52edd068"). InnerVolumeSpecName "kube-api-access-zsz64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:42:25.190471 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.190432 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-service-ca\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:42:25.190471 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.190464 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-config\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:42:25.190471 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.190475 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-serving-cert\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:42:25.190471 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.190484 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsz64\" (UniqueName: \"kubernetes.io/projected/975fc5fb-7c9c-4bff-ad02-71bc52edd068-kube-api-access-zsz64\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:42:25.190725 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.190493 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/975fc5fb-7c9c-4bff-ad02-71bc52edd068-oauth-serving-cert\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:42:25.190725 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.190501 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/975fc5fb-7c9c-4bff-ad02-71bc52edd068-console-oauth-config\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:42:25.364656 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.364624 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664869df9f-v57vl"] Sep 15 06:42:25.368137 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:25.368107 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-664869df9f-v57vl"] Sep 15 06:42:26.860486 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:26.860448 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="975fc5fb-7c9c-4bff-ad02-71bc52edd068" path="/var/lib/kubelet/pods/975fc5fb-7c9c-4bff-ad02-71bc52edd068/volumes" Sep 15 06:42:50.696857 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:50.696817 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:42:50.699017 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:50.698998 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532a12f1-78ac-4e4d-a02d-6be6c808825f-metrics-certs\") pod \"network-metrics-daemon-ns6cw\" (UID: \"532a12f1-78ac-4e4d-a02d-6be6c808825f\") " pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:42:50.862990 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:50.859627 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5bkjm\"" Sep 15 06:42:50.866733 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:50.866708 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ns6cw" Sep 15 06:42:51.133698 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:51.133672 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ns6cw"] Sep 15 06:42:51.136414 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:42:51.136377 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532a12f1_78ac_4e4d_a02d_6be6c808825f.slice/crio-b9689d1aeb4a621726d6437b1ea407e1d785490c066723bfb9c7d98db0f108d3 WatchSource:0}: Error finding container b9689d1aeb4a621726d6437b1ea407e1d785490c066723bfb9c7d98db0f108d3: Status 404 returned error can't find the container with id b9689d1aeb4a621726d6437b1ea407e1d785490c066723bfb9c7d98db0f108d3 Sep 15 06:42:52.146043 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:52.145833 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ns6cw" event={"ID":"532a12f1-78ac-4e4d-a02d-6be6c808825f","Type":"ContainerStarted","Data":"efdc410225d556631a65b44a8217216f92b554e31204ccc5861a2bd17ec42705"} Sep 15 06:42:52.146043 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:52.145943 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ns6cw" event={"ID":"532a12f1-78ac-4e4d-a02d-6be6c808825f","Type":"ContainerStarted","Data":"b9689d1aeb4a621726d6437b1ea407e1d785490c066723bfb9c7d98db0f108d3"} Sep 15 06:42:53.152702 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:53.152665 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ns6cw" event={"ID":"532a12f1-78ac-4e4d-a02d-6be6c808825f","Type":"ContainerStarted","Data":"ede8bcff1ee6190f2a2a714e13133cbab60cae307a27df892916026db0d78741"} Sep 15 06:42:53.168685 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:53.168632 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ns6cw" podStartSLOduration=376.290968195 podStartE2EDuration="6m17.168615757s" totalImagesPullingTime="877.647562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:42:53.167110436 +0000 UTC m=+376.721767859" watchObservedRunningTime="2026-09-15 06:42:53.168615757 +0000 UTC m=+376.723273179" Sep 15 06:42:56.954462 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.954428 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d79bbf55-7ngvq"] Sep 15 06:42:56.954835 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.954671 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="975fc5fb-7c9c-4bff-ad02-71bc52edd068" containerName="console" Sep 15 06:42:56.954835 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.954682 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="975fc5fb-7c9c-4bff-ad02-71bc52edd068" containerName="console" Sep 15 06:42:56.954835 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.954733 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="975fc5fb-7c9c-4bff-ad02-71bc52edd068" containerName="console" Sep 15 06:42:56.956472 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.956457 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:56.959025 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.958995 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 15 06:42:56.959175 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.959002 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 15 06:42:56.959175 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.959099 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 15 06:42:56.960246 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.960231 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 15 06:42:56.960321 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.960306 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 15 06:42:56.960451 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.960434 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ksjg5\"" Sep 15 06:42:56.964992 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.964956 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 15 06:42:56.967597 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:56.967576 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d79bbf55-7ngvq"] Sep 15 06:42:57.042915 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.042873 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-config\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.042915 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.042911 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-oauth-serving-cert\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.043190 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.042936 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-service-ca\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.043190 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.042957 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-trusted-ca-bundle\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.043190 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.043055 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-serving-cert\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.043190 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.043077 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vvm4\" (UniqueName: \"kubernetes.io/projected/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-kube-api-access-9vvm4\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.043190 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.043101 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-oauth-config\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.144042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144000 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-serving-cert\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.144042 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144041 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vvm4\" (UniqueName: \"kubernetes.io/projected/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-kube-api-access-9vvm4\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.144263 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144059 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-oauth-config\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.144263 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144092 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-config\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.144263 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144106 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-oauth-serving-cert\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.144263 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144122 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-service-ca\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.144263 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144135 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-trusted-ca-bundle\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.144926 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144895 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-config\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.145059 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144895 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-oauth-serving-cert\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.145059 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.144953 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-service-ca\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.145059 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.145047 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-trusted-ca-bundle\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.146411 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.146387 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-oauth-config\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.147026 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.147007 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-serving-cert\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.152492 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.152473 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vvm4\" (UniqueName: \"kubernetes.io/projected/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-kube-api-access-9vvm4\") pod \"console-7d79bbf55-7ngvq\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.266014 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.265904 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:42:57.547191 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:57.547165 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d79bbf55-7ngvq"] Sep 15 06:42:57.550082 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:42:57.550053 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f9bb8d_85cf_4b86_97d9_fc00ba2f01eb.slice/crio-c4f2cfffc38371ef12a2d547dfeb91e7520f89600da0457d609e24e926e38a5d WatchSource:0}: Error finding container c4f2cfffc38371ef12a2d547dfeb91e7520f89600da0457d609e24e926e38a5d: Status 404 returned error can't find the container with id c4f2cfffc38371ef12a2d547dfeb91e7520f89600da0457d609e24e926e38a5d Sep 15 06:42:58.171946 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:58.171915 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d79bbf55-7ngvq" event={"ID":"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb","Type":"ContainerStarted","Data":"65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724"} Sep 15 06:42:58.172303 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:58.171961 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d79bbf55-7ngvq" event={"ID":"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb","Type":"ContainerStarted","Data":"c4f2cfffc38371ef12a2d547dfeb91e7520f89600da0457d609e24e926e38a5d"} Sep 15 06:42:58.189907 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:42:58.189857 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d79bbf55-7ngvq" podStartSLOduration=2.189842483 podStartE2EDuration="2.189842483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:42:58.189431888 +0000 UTC m=+381.744089312" watchObservedRunningTime="2026-09-15 06:42:58.189842483 +0000 UTC m=+381.744499903" Sep 15 06:43:07.266292 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:43:07.266239 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:43:07.266292 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:43:07.266311 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:43:07.271079 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:43:07.271058 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:43:08.214841 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:43:08.214810 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:44:15.190386 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:15.190354 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d79bbf55-7ngvq"] Sep 15 06:44:40.213185 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.213143 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d79bbf55-7ngvq" podUID="76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" containerName="console" containerID="cri-o://65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724" gracePeriod=15 Sep 15 06:44:40.451677 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.451654 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:44:40.510170 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510105 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-oauth-config\") pod \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " Sep 15 06:44:40.510170 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510138 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-config\") pod \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " Sep 15 06:44:40.510170 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510162 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-trusted-ca-bundle\") pod \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " Sep 15 06:44:40.510402 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510185 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-serving-cert\") pod \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " Sep 15 06:44:40.510402 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510300 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-oauth-serving-cert\") pod \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " Sep 15 06:44:40.510402 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510335 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vvm4\" (UniqueName: \"kubernetes.io/projected/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-kube-api-access-9vvm4\") pod \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " Sep 15 06:44:40.510402 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510361 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-service-ca\") pod \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\" (UID: \"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb\") " Sep 15 06:44:40.510604 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510555 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-config" (OuterVolumeSpecName: "console-config") pod "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" (UID: "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:44:40.510604 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510578 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" (UID: "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:44:40.510683 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510629 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" (UID: "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:44:40.510857 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.510833 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-service-ca" (OuterVolumeSpecName: "service-ca") pod "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" (UID: "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 06:44:40.512198 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.512172 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" (UID: "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:44:40.512467 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.512447 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" (UID: "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 06:44:40.512539 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.512462 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-kube-api-access-9vvm4" (OuterVolumeSpecName: "kube-api-access-9vvm4") pod "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" (UID: "76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb"). InnerVolumeSpecName "kube-api-access-9vvm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:44:40.544592 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.544564 2552 generic.go:356] "Generic (PLEG): container finished" podID="76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" containerID="65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724" exitCode=0 Sep 15 06:44:40.544719 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.544612 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d79bbf55-7ngvq" Sep 15 06:44:40.544719 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.544613 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d79bbf55-7ngvq" event={"ID":"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb","Type":"ContainerDied","Data":"65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724"} Sep 15 06:44:40.544719 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.544659 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d79bbf55-7ngvq" event={"ID":"76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb","Type":"ContainerDied","Data":"c4f2cfffc38371ef12a2d547dfeb91e7520f89600da0457d609e24e926e38a5d"} Sep 15 06:44:40.544719 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.544677 2552 scope.go:117] "RemoveContainer" containerID="65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724" Sep 15 06:44:40.552622 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.552606 2552 scope.go:117] "RemoveContainer" containerID="65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724" Sep 15 06:44:40.552862 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:44:40.552847 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724\": container with ID starting with 65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724 not found: ID does not exist" containerID="65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724" Sep 15 06:44:40.552898 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.552868 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724"} err="failed to get container status \"65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724\": rpc error: code = NotFound desc = could not find container \"65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724\": container with ID starting with 65cde8a69d99e2efd45bec101e1b038498957bff6412e913a88718cfcffd2724 not found: ID does not exist" Sep 15 06:44:40.564128 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.564103 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d79bbf55-7ngvq"] Sep 15 06:44:40.566709 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.566689 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d79bbf55-7ngvq"] Sep 15 06:44:40.611477 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.611450 2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-oauth-config\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:44:40.611477 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.611474 2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-config\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:44:40.611601 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.611488 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-trusted-ca-bundle\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:44:40.611601 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.611500 2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-console-serving-cert\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:44:40.611601 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.611515 2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-oauth-serving-cert\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:44:40.611601 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.611529 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vvm4\" (UniqueName: \"kubernetes.io/projected/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-kube-api-access-9vvm4\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:44:40.611601 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.611542 2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb-service-ca\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:44:40.864808 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:44:40.862662 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" path="/var/lib/kubelet/pods/76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb/volumes" Sep 15 06:46:47.159730 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.159695 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4"] Sep 15 06:46:47.160142 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.159954 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" containerName="console" Sep 15 06:46:47.160142 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.159977 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" containerName="console" Sep 15 06:46:47.160142 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.160040 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f9bb8d-85cf-4b86-97d9-fc00ba2f01eb" containerName="console" Sep 15 06:46:47.162711 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.162692 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 06:46:47.165412 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.165383 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 15 06:46:47.165488 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.165414 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 15 06:46:47.166517 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.166498 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-q8rkv\"" Sep 15 06:46:47.180726 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.180705 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4"] Sep 15 06:46:47.317229 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.317198 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7db\" (UniqueName: \"kubernetes.io/projected/39101fe5-602d-4bff-9bf6-01849264a1e8-kube-api-access-6v7db\") pod \"feast-operator-controller-manager-f76d56774-x7pf4\" (UID: \"39101fe5-602d-4bff-9bf6-01849264a1e8\") " pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 06:46:47.417577 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.417502 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7db\" (UniqueName: \"kubernetes.io/projected/39101fe5-602d-4bff-9bf6-01849264a1e8-kube-api-access-6v7db\") pod \"feast-operator-controller-manager-f76d56774-x7pf4\" (UID: \"39101fe5-602d-4bff-9bf6-01849264a1e8\") " pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 06:46:47.425298 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.425273 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7db\" (UniqueName: \"kubernetes.io/projected/39101fe5-602d-4bff-9bf6-01849264a1e8-kube-api-access-6v7db\") pod \"feast-operator-controller-manager-f76d56774-x7pf4\" (UID: \"39101fe5-602d-4bff-9bf6-01849264a1e8\") " pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 06:46:47.472163 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.472142 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 06:46:47.737098 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.737071 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4"] Sep 15 06:46:47.739579 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:46:47.739549 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39101fe5_602d_4bff_9bf6_01849264a1e8.slice/crio-5098f2a7d6cb7464da6a5c31f1c0fa762027a744d48bc1a4b8cab2d2179abf03 WatchSource:0}: Error finding container 5098f2a7d6cb7464da6a5c31f1c0fa762027a744d48bc1a4b8cab2d2179abf03: Status 404 returned error can't find the container with id 5098f2a7d6cb7464da6a5c31f1c0fa762027a744d48bc1a4b8cab2d2179abf03 Sep 15 06:46:47.741524 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.741507 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 06:46:47.984738 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:47.984709 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" event={"ID":"39101fe5-602d-4bff-9bf6-01849264a1e8","Type":"ContainerStarted","Data":"5098f2a7d6cb7464da6a5c31f1c0fa762027a744d48bc1a4b8cab2d2179abf03"} Sep 15 06:46:50.997398 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:50.997361 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" event={"ID":"39101fe5-602d-4bff-9bf6-01849264a1e8","Type":"ContainerStarted","Data":"579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9"} Sep 15 06:46:50.997783 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:50.997443 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 06:46:51.013643 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:46:51.013596 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" podStartSLOduration=1.446897119 podStartE2EDuration="4.013582349s" totalImagesPullingTime="2.56668523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:46:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:46:51.012940676 +0000 UTC m=+614.567598107" watchObservedRunningTime="2026-09-15 06:46:51.013582349 +0000 UTC m=+614.568239773" Sep 15 06:47:02.003722 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:47:02.003641 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 06:53:06.543592 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.543516 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-tlr4j"] Sep 15 06:53:06.546941 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.546919 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:53:06.549469 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.549448 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 15 06:53:06.549818 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.549794 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-t7msp\"" Sep 15 06:53:06.550647 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.550625 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 15 06:53:06.550865 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.550848 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 15 06:53:06.562364 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.562331 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tlr4j"] Sep 15 06:53:06.595316 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.595287 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7km2\" (UniqueName: \"kubernetes.io/projected/ae0f0cf8-ff9a-476d-8c49-40575702ec48-kube-api-access-l7km2\") pod \"postgres-864846d4f5-tlr4j\" (UID: \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\") " pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:53:06.595460 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.595321 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ae0f0cf8-ff9a-476d-8c49-40575702ec48-postgresdata\") pod \"postgres-864846d4f5-tlr4j\" (UID: \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\") " pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:53:06.696397 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.696350 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7km2\" (UniqueName: \"kubernetes.io/projected/ae0f0cf8-ff9a-476d-8c49-40575702ec48-kube-api-access-l7km2\") pod \"postgres-864846d4f5-tlr4j\" (UID: \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\") " pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:53:06.696397 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.696401 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ae0f0cf8-ff9a-476d-8c49-40575702ec48-postgresdata\") pod \"postgres-864846d4f5-tlr4j\" (UID: \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\") " pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:53:06.696721 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.696707 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ae0f0cf8-ff9a-476d-8c49-40575702ec48-postgresdata\") pod \"postgres-864846d4f5-tlr4j\" (UID: \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\") " pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:53:06.704211 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.704185 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7km2\" (UniqueName: \"kubernetes.io/projected/ae0f0cf8-ff9a-476d-8c49-40575702ec48-kube-api-access-l7km2\") pod \"postgres-864846d4f5-tlr4j\" (UID: \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\") " pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:53:06.864474 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:06.864401 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:53:07.128558 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:07.128533 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tlr4j"] Sep 15 06:53:07.130432 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:53:07.130405 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae0f0cf8_ff9a_476d_8c49_40575702ec48.slice/crio-70b2706f02c6809e8e1933b851d805cdfdfde5c588dee6c1fb434459d0ef9fa4 WatchSource:0}: Error finding container 70b2706f02c6809e8e1933b851d805cdfdfde5c588dee6c1fb434459d0ef9fa4: Status 404 returned error can't find the container with id 70b2706f02c6809e8e1933b851d805cdfdfde5c588dee6c1fb434459d0ef9fa4 Sep 15 06:53:07.132242 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:07.132217 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 06:53:07.327770 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:07.327735 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-tlr4j" event={"ID":"ae0f0cf8-ff9a-476d-8c49-40575702ec48","Type":"ContainerStarted","Data":"70b2706f02c6809e8e1933b851d805cdfdfde5c588dee6c1fb434459d0ef9fa4"} Sep 15 06:53:14.356652 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:14.356611 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-tlr4j" event={"ID":"ae0f0cf8-ff9a-476d-8c49-40575702ec48","Type":"ContainerStarted","Data":"887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060"} Sep 15 06:53:14.371738 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:14.371693 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-tlr4j" podStartSLOduration=1.751345854 podStartE2EDuration="8.371677765s" totalImagesPullingTime="6.620331911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:53:14.370517079 +0000 UTC m=+997.925174507" watchObservedRunningTime="2026-09-15 06:53:14.371677765 +0000 UTC m=+997.926335188" Sep 15 06:53:44.021744 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.021698 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-dzrjb"] Sep 15 06:53:44.024843 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.024827 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-dzrjb" Sep 15 06:53:44.027385 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.027365 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-4jwsc\"" Sep 15 06:53:44.027385 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.027373 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 15 06:53:44.031638 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.031500 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-dzrjb"] Sep 15 06:53:44.079209 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.079177 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn29n\" (UniqueName: \"kubernetes.io/projected/cba8f7df-b132-4101-837f-5de6fc3656d5-kube-api-access-gn29n\") pod \"feast-test-apply-dzrjb\" (UID: \"cba8f7df-b132-4101-837f-5de6fc3656d5\") " pod="test-ns-feast/feast-test-apply-dzrjb" Sep 15 06:53:44.179675 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.179634 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn29n\" (UniqueName: \"kubernetes.io/projected/cba8f7df-b132-4101-837f-5de6fc3656d5-kube-api-access-gn29n\") pod \"feast-test-apply-dzrjb\" (UID: \"cba8f7df-b132-4101-837f-5de6fc3656d5\") " pod="test-ns-feast/feast-test-apply-dzrjb" Sep 15 06:53:44.188075 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.188044 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn29n\" (UniqueName: \"kubernetes.io/projected/cba8f7df-b132-4101-837f-5de6fc3656d5-kube-api-access-gn29n\") pod \"feast-test-apply-dzrjb\" (UID: \"cba8f7df-b132-4101-837f-5de6fc3656d5\") " pod="test-ns-feast/feast-test-apply-dzrjb" Sep 15 06:53:44.334364 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.334277 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-dzrjb" Sep 15 06:53:44.600302 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:44.600169 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-dzrjb"] Sep 15 06:53:44.602605 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:53:44.602578 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcba8f7df_b132_4101_837f_5de6fc3656d5.slice/crio-665e563140cbf2af58ee64a67bdbdd38b6761f6609df9d95c679c2d183708086 WatchSource:0}: Error finding container 665e563140cbf2af58ee64a67bdbdd38b6761f6609df9d95c679c2d183708086: Status 404 returned error can't find the container with id 665e563140cbf2af58ee64a67bdbdd38b6761f6609df9d95c679c2d183708086 Sep 15 06:53:45.467112 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:45.467078 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-dzrjb" event={"ID":"cba8f7df-b132-4101-837f-5de6fc3656d5","Type":"ContainerStarted","Data":"665e563140cbf2af58ee64a67bdbdd38b6761f6609df9d95c679c2d183708086"} Sep 15 06:53:52.498155 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:52.498119 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-dzrjb" event={"ID":"cba8f7df-b132-4101-837f-5de6fc3656d5","Type":"ContainerStarted","Data":"6af19b6dbc2fb6d87b988df02ff9663e55521e73d801e5734b9d5558cb09f096"} Sep 15 06:53:58.520660 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:58.520623 2552 generic.go:356] "Generic (PLEG): container finished" podID="cba8f7df-b132-4101-837f-5de6fc3656d5" containerID="6af19b6dbc2fb6d87b988df02ff9663e55521e73d801e5734b9d5558cb09f096" exitCode=0 Sep 15 06:53:58.521120 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:58.520676 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-dzrjb" event={"ID":"cba8f7df-b132-4101-837f-5de6fc3656d5","Type":"ContainerDied","Data":"6af19b6dbc2fb6d87b988df02ff9663e55521e73d801e5734b9d5558cb09f096"} Sep 15 06:53:59.526497 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:59.526463 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-dzrjb" event={"ID":"cba8f7df-b132-4101-837f-5de6fc3656d5","Type":"ContainerStarted","Data":"a5da4a0e77ba111ec719595ce6f9456a33164da9d400d23240b2148116c75371"} Sep 15 06:53:59.541145 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:53:59.541083 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-dzrjb" podStartSLOduration=1.830055202 podStartE2EDuration="15.54106539s" totalImagesPullingTime="7.174461135s" totalInitContainerRuntime="6.536549053s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:53:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:53:59.540932834 +0000 UTC m=+1043.095590257" watchObservedRunningTime="2026-09-15 06:53:59.54106539 +0000 UTC m=+1043.095722814" Sep 15 06:54:08.562750 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:08.562718 2552 generic.go:356] "Generic (PLEG): container finished" podID="cba8f7df-b132-4101-837f-5de6fc3656d5" containerID="a5da4a0e77ba111ec719595ce6f9456a33164da9d400d23240b2148116c75371" exitCode=0 Sep 15 06:54:08.563138 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:08.562779 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-dzrjb" event={"ID":"cba8f7df-b132-4101-837f-5de6fc3656d5","Type":"ContainerDied","Data":"a5da4a0e77ba111ec719595ce6f9456a33164da9d400d23240b2148116c75371"} Sep 15 06:54:09.695140 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:09.695110 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-dzrjb" Sep 15 06:54:09.782946 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:09.782913 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn29n\" (UniqueName: \"kubernetes.io/projected/cba8f7df-b132-4101-837f-5de6fc3656d5-kube-api-access-gn29n\") pod \"cba8f7df-b132-4101-837f-5de6fc3656d5\" (UID: \"cba8f7df-b132-4101-837f-5de6fc3656d5\") " Sep 15 06:54:09.785065 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:09.785036 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cba8f7df-b132-4101-837f-5de6fc3656d5-kube-api-access-gn29n" (OuterVolumeSpecName: "kube-api-access-gn29n") pod "cba8f7df-b132-4101-837f-5de6fc3656d5" (UID: "cba8f7df-b132-4101-837f-5de6fc3656d5"). InnerVolumeSpecName "kube-api-access-gn29n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:54:09.883821 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:09.883742 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn29n\" (UniqueName: \"kubernetes.io/projected/cba8f7df-b132-4101-837f-5de6fc3656d5-kube-api-access-gn29n\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:54:10.572781 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:10.572746 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-dzrjb" event={"ID":"cba8f7df-b132-4101-837f-5de6fc3656d5","Type":"ContainerDied","Data":"665e563140cbf2af58ee64a67bdbdd38b6761f6609df9d95c679c2d183708086"} Sep 15 06:54:10.572781 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:10.572783 2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="665e563140cbf2af58ee64a67bdbdd38b6761f6609df9d95c679c2d183708086" Sep 15 06:54:10.572992 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:10.572786 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-dzrjb" Sep 15 06:54:10.735273 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:10.735249 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-dzrjb_cba8f7df-b132-4101-837f-5de6fc3656d5/feast-0/0.log" Sep 15 06:54:10.742067 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:10.742034 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-dzrjb_cba8f7df-b132-4101-837f-5de6fc3656d5/feast-1/0.log" Sep 15 06:54:48.314182 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.314104 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-dzrjb"] Sep 15 06:54:48.325386 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.325359 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-dzrjb"] Sep 15 06:54:48.329818 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.329795 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tlr4j"] Sep 15 06:54:48.329996 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.329944 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-tlr4j" podUID="ae0f0cf8-ff9a-476d-8c49-40575702ec48" containerName="postgres" containerID="cri-o://887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060" gracePeriod=30 Sep 15 06:54:48.567310 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.567256 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:54:48.662083 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.662050 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7km2\" (UniqueName: \"kubernetes.io/projected/ae0f0cf8-ff9a-476d-8c49-40575702ec48-kube-api-access-l7km2\") pod \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\" (UID: \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\") " Sep 15 06:54:48.662252 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.662097 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ae0f0cf8-ff9a-476d-8c49-40575702ec48-postgresdata\") pod \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\" (UID: \"ae0f0cf8-ff9a-476d-8c49-40575702ec48\") " Sep 15 06:54:48.663980 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.663936 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0f0cf8-ff9a-476d-8c49-40575702ec48-kube-api-access-l7km2" (OuterVolumeSpecName: "kube-api-access-l7km2") pod "ae0f0cf8-ff9a-476d-8c49-40575702ec48" (UID: "ae0f0cf8-ff9a-476d-8c49-40575702ec48"). InnerVolumeSpecName "kube-api-access-l7km2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 06:54:48.691568 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.691534 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0f0cf8-ff9a-476d-8c49-40575702ec48-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "ae0f0cf8-ff9a-476d-8c49-40575702ec48" (UID: "ae0f0cf8-ff9a-476d-8c49-40575702ec48"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 06:54:48.716623 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.716596 2552 generic.go:356] "Generic (PLEG): container finished" podID="ae0f0cf8-ff9a-476d-8c49-40575702ec48" containerID="887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060" exitCode=0 Sep 15 06:54:48.716724 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.716641 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-tlr4j" Sep 15 06:54:48.716724 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.716685 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-tlr4j" event={"ID":"ae0f0cf8-ff9a-476d-8c49-40575702ec48","Type":"ContainerDied","Data":"887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060"} Sep 15 06:54:48.716806 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.716729 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-tlr4j" event={"ID":"ae0f0cf8-ff9a-476d-8c49-40575702ec48","Type":"ContainerDied","Data":"70b2706f02c6809e8e1933b851d805cdfdfde5c588dee6c1fb434459d0ef9fa4"} Sep 15 06:54:48.716806 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.716745 2552 scope.go:117] "RemoveContainer" containerID="887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060" Sep 15 06:54:48.724606 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.724582 2552 scope.go:117] "RemoveContainer" containerID="887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060" Sep 15 06:54:48.724899 ip-10-0-133-232 kubenswrapper[2552]: E0915 06:54:48.724878 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060\": container with ID starting with 887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060 not found: ID does not exist" containerID="887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060" Sep 15 06:54:48.724989 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.724905 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060"} err="failed to get container status \"887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060\": rpc error: code = NotFound desc = could not find container \"887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060\": container with ID starting with 887157617c0f01a1165c886cedd52930eafd325c3a10417213a59801aa701060 not found: ID does not exist" Sep 15 06:54:48.736766 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.736745 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tlr4j"] Sep 15 06:54:48.740220 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.740199 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-tlr4j"] Sep 15 06:54:48.763383 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.763354 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7km2\" (UniqueName: \"kubernetes.io/projected/ae0f0cf8-ff9a-476d-8c49-40575702ec48-kube-api-access-l7km2\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:54:48.763383 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.763382 2552 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ae0f0cf8-ff9a-476d-8c49-40575702ec48-postgresdata\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 06:54:48.860663 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.860592 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae0f0cf8-ff9a-476d-8c49-40575702ec48" path="/var/lib/kubelet/pods/ae0f0cf8-ff9a-476d-8c49-40575702ec48/volumes" Sep 15 06:54:48.860908 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:54:48.860894 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cba8f7df-b132-4101-837f-5de6fc3656d5" path="/var/lib/kubelet/pods/cba8f7df-b132-4101-837f-5de6fc3656d5/volumes" Sep 15 06:57:47.896240 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896162 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jp56l"] Sep 15 06:57:47.898436 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896444 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cba8f7df-b132-4101-837f-5de6fc3656d5" containerName="feast-0" Sep 15 06:57:47.898436 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896458 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="cba8f7df-b132-4101-837f-5de6fc3656d5" containerName="feast-0" Sep 15 06:57:47.898436 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896473 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0f0cf8-ff9a-476d-8c49-40575702ec48" containerName="postgres" Sep 15 06:57:47.898436 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896479 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0f0cf8-ff9a-476d-8c49-40575702ec48" containerName="postgres" Sep 15 06:57:47.898436 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896486 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cba8f7df-b132-4101-837f-5de6fc3656d5" containerName="feast-1" Sep 15 06:57:47.898436 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896491 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="cba8f7df-b132-4101-837f-5de6fc3656d5" containerName="feast-1" Sep 15 06:57:47.898436 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896542 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="cba8f7df-b132-4101-837f-5de6fc3656d5" containerName="feast-1" Sep 15 06:57:47.898436 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.896549 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0f0cf8-ff9a-476d-8c49-40575702ec48" containerName="postgres" Sep 15 06:57:47.899259 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.899243 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" Sep 15 06:57:47.901819 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.901795 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 15 06:57:47.901947 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.901901 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 15 06:57:47.902033 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.901859 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-xqrdp\"" Sep 15 06:57:47.910112 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:47.910077 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jp56l"] Sep 15 06:57:48.026440 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:48.026415 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mm87\" (UniqueName: \"kubernetes.io/projected/c5d02e26-d0c0-48ed-b8be-ec4287f6986e-kube-api-access-6mm87\") pod \"redis-69bbd798b6-jp56l\" (UID: \"c5d02e26-d0c0-48ed-b8be-ec4287f6986e\") " pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" Sep 15 06:57:48.127144 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:48.127113 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mm87\" (UniqueName: \"kubernetes.io/projected/c5d02e26-d0c0-48ed-b8be-ec4287f6986e-kube-api-access-6mm87\") pod \"redis-69bbd798b6-jp56l\" (UID: \"c5d02e26-d0c0-48ed-b8be-ec4287f6986e\") " pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" Sep 15 06:57:48.135471 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:48.135443 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mm87\" (UniqueName: \"kubernetes.io/projected/c5d02e26-d0c0-48ed-b8be-ec4287f6986e-kube-api-access-6mm87\") pod \"redis-69bbd798b6-jp56l\" (UID: \"c5d02e26-d0c0-48ed-b8be-ec4287f6986e\") " pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" Sep 15 06:57:48.209072 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:48.209044 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" Sep 15 06:57:48.473584 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:48.473563 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jp56l"] Sep 15 06:57:48.476140 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:57:48.476111 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5d02e26_d0c0_48ed_b8be_ec4287f6986e.slice/crio-67e66e7ff6d5e41e9a47883e6a1d2427c24e96567e44dce2ed9bca4cd625a07a WatchSource:0}: Error finding container 67e66e7ff6d5e41e9a47883e6a1d2427c24e96567e44dce2ed9bca4cd625a07a: Status 404 returned error can't find the container with id 67e66e7ff6d5e41e9a47883e6a1d2427c24e96567e44dce2ed9bca4cd625a07a Sep 15 06:57:49.364913 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:49.364870 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" event={"ID":"c5d02e26-d0c0-48ed-b8be-ec4287f6986e","Type":"ContainerStarted","Data":"67e66e7ff6d5e41e9a47883e6a1d2427c24e96567e44dce2ed9bca4cd625a07a"} Sep 15 06:57:50.369796 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:50.369760 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" event={"ID":"c5d02e26-d0c0-48ed-b8be-ec4287f6986e","Type":"ContainerStarted","Data":"8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4"} Sep 15 06:57:50.383448 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:50.383402 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" podStartSLOduration=2.045553533 podStartE2EDuration="3.383383831s" totalImagesPullingTime="1.337830298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:57:50.382787032 +0000 UTC m=+1273.937444455" watchObservedRunningTime="2026-09-15 06:57:50.383383831 +0000 UTC m=+1273.938041258" Sep 15 06:57:53.291159 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.291129 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-zqt2t"] Sep 15 06:57:53.294399 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.294380 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 06:57:53.296768 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.296748 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 15 06:57:53.300575 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.300551 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-zqt2t"] Sep 15 06:57:53.470793 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.470756 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhgk2\" (UniqueName: \"kubernetes.io/projected/4680c9ea-077e-4e2e-90d6-169088b146a9-kube-api-access-fhgk2\") pod \"postgres-864846d4f5-zqt2t\" (UID: \"4680c9ea-077e-4e2e-90d6-169088b146a9\") " pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 06:57:53.470958 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.470804 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4680c9ea-077e-4e2e-90d6-169088b146a9-postgresdata\") pod \"postgres-864846d4f5-zqt2t\" (UID: \"4680c9ea-077e-4e2e-90d6-169088b146a9\") " pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 06:57:53.571396 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.571320 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4680c9ea-077e-4e2e-90d6-169088b146a9-postgresdata\") pod \"postgres-864846d4f5-zqt2t\" (UID: \"4680c9ea-077e-4e2e-90d6-169088b146a9\") " pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 06:57:53.571483 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.571416 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhgk2\" (UniqueName: \"kubernetes.io/projected/4680c9ea-077e-4e2e-90d6-169088b146a9-kube-api-access-fhgk2\") pod \"postgres-864846d4f5-zqt2t\" (UID: \"4680c9ea-077e-4e2e-90d6-169088b146a9\") " pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 06:57:53.571738 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.571722 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4680c9ea-077e-4e2e-90d6-169088b146a9-postgresdata\") pod \"postgres-864846d4f5-zqt2t\" (UID: \"4680c9ea-077e-4e2e-90d6-169088b146a9\") " pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 06:57:53.579148 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.579129 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhgk2\" (UniqueName: \"kubernetes.io/projected/4680c9ea-077e-4e2e-90d6-169088b146a9-kube-api-access-fhgk2\") pod \"postgres-864846d4f5-zqt2t\" (UID: \"4680c9ea-077e-4e2e-90d6-169088b146a9\") " pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 06:57:53.604893 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.604871 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 06:57:53.869395 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:53.869321 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-zqt2t"] Sep 15 06:57:53.872324 ip-10-0-133-232 kubenswrapper[2552]: W0915 06:57:53.872300 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4680c9ea_077e_4e2e_90d6_169088b146a9.slice/crio-3812d42d9ef20296c571a669a09c59e3d4a3485f0de296d057addd9eba4cef92 WatchSource:0}: Error finding container 3812d42d9ef20296c571a669a09c59e3d4a3485f0de296d057addd9eba4cef92: Status 404 returned error can't find the container with id 3812d42d9ef20296c571a669a09c59e3d4a3485f0de296d057addd9eba4cef92 Sep 15 06:57:54.385790 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:54.385747 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" event={"ID":"4680c9ea-077e-4e2e-90d6-169088b146a9","Type":"ContainerStarted","Data":"e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff"} Sep 15 06:57:54.385790 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:54.385795 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" event={"ID":"4680c9ea-077e-4e2e-90d6-169088b146a9","Type":"ContainerStarted","Data":"3812d42d9ef20296c571a669a09c59e3d4a3485f0de296d057addd9eba4cef92"} Sep 15 06:57:54.401122 ip-10-0-133-232 kubenswrapper[2552]: I0915 06:57:54.401070 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" podStartSLOduration=1.401053444 podStartE2EDuration="1.401053444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 06:57:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 06:57:54.399252391 +0000 UTC m=+1277.953909815" watchObservedRunningTime="2026-09-15 06:57:54.401053444 +0000 UTC m=+1277.955710867" Sep 15 07:00:18.499285 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.499246 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4"] Sep 15 07:00:18.499855 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.499568 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" podUID="39101fe5-602d-4bff-9bf6-01849264a1e8" containerName="manager" containerID="cri-o://579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9" gracePeriod=10 Sep 15 07:00:18.739349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.739323 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 07:00:18.834316 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.834233 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v7db\" (UniqueName: \"kubernetes.io/projected/39101fe5-602d-4bff-9bf6-01849264a1e8-kube-api-access-6v7db\") pod \"39101fe5-602d-4bff-9bf6-01849264a1e8\" (UID: \"39101fe5-602d-4bff-9bf6-01849264a1e8\") " Sep 15 07:00:18.836336 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.836297 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39101fe5-602d-4bff-9bf6-01849264a1e8-kube-api-access-6v7db" (OuterVolumeSpecName: "kube-api-access-6v7db") pod "39101fe5-602d-4bff-9bf6-01849264a1e8" (UID: "39101fe5-602d-4bff-9bf6-01849264a1e8"). InnerVolumeSpecName "kube-api-access-6v7db". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:00:18.899796 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.899766 2552 generic.go:356] "Generic (PLEG): container finished" podID="39101fe5-602d-4bff-9bf6-01849264a1e8" containerID="579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9" exitCode=0 Sep 15 07:00:18.899949 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.899807 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" Sep 15 07:00:18.899949 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.899832 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" event={"ID":"39101fe5-602d-4bff-9bf6-01849264a1e8","Type":"ContainerDied","Data":"579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9"} Sep 15 07:00:18.899949 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.899872 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4" event={"ID":"39101fe5-602d-4bff-9bf6-01849264a1e8","Type":"ContainerDied","Data":"5098f2a7d6cb7464da6a5c31f1c0fa762027a744d48bc1a4b8cab2d2179abf03"} Sep 15 07:00:18.899949 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.899892 2552 scope.go:117] "RemoveContainer" containerID="579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9" Sep 15 07:00:18.909673 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.909654 2552 scope.go:117] "RemoveContainer" containerID="579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9" Sep 15 07:00:18.909943 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:00:18.909922 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9\": container with ID starting with 579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9 not found: ID does not exist" containerID="579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9" Sep 15 07:00:18.910026 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.909951 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9"} err="failed to get container status \"579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9\": rpc error: code = NotFound desc = could not find container \"579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9\": container with ID starting with 579147c3136b6b8dca70fff8359e211624991443d9b7b5279af675a21d0dc9c9 not found: ID does not exist" Sep 15 07:00:18.919051 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.919028 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4"] Sep 15 07:00:18.921019 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.920998 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f76d56774-x7pf4"] Sep 15 07:00:18.935366 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:18.935340 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v7db\" (UniqueName: \"kubernetes.io/projected/39101fe5-602d-4bff-9bf6-01849264a1e8-kube-api-access-6v7db\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:00:20.861605 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:20.861571 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39101fe5-602d-4bff-9bf6-01849264a1e8" path="/var/lib/kubelet/pods/39101fe5-602d-4bff-9bf6-01849264a1e8/volumes" Sep 15 07:00:21.077698 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.077662 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-zqt2t"] Sep 15 07:00:21.077866 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.077834 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" podUID="4680c9ea-077e-4e2e-90d6-169088b146a9" containerName="postgres" containerID="cri-o://e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff" gracePeriod=30 Sep 15 07:00:21.082897 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.082872 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jp56l"] Sep 15 07:00:21.083049 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.083031 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" podUID="c5d02e26-d0c0-48ed-b8be-ec4287f6986e" containerName="redis" containerID="cri-o://8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4" gracePeriod=30 Sep 15 07:00:21.319305 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.319280 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 07:00:21.336833 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.336812 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" Sep 15 07:00:21.454387 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.454358 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mm87\" (UniqueName: \"kubernetes.io/projected/c5d02e26-d0c0-48ed-b8be-ec4287f6986e-kube-api-access-6mm87\") pod \"c5d02e26-d0c0-48ed-b8be-ec4287f6986e\" (UID: \"c5d02e26-d0c0-48ed-b8be-ec4287f6986e\") " Sep 15 07:00:21.454529 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.454417 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4680c9ea-077e-4e2e-90d6-169088b146a9-postgresdata\") pod \"4680c9ea-077e-4e2e-90d6-169088b146a9\" (UID: \"4680c9ea-077e-4e2e-90d6-169088b146a9\") " Sep 15 07:00:21.454529 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.454459 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhgk2\" (UniqueName: \"kubernetes.io/projected/4680c9ea-077e-4e2e-90d6-169088b146a9-kube-api-access-fhgk2\") pod \"4680c9ea-077e-4e2e-90d6-169088b146a9\" (UID: \"4680c9ea-077e-4e2e-90d6-169088b146a9\") " Sep 15 07:00:21.456473 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.456440 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d02e26-d0c0-48ed-b8be-ec4287f6986e-kube-api-access-6mm87" (OuterVolumeSpecName: "kube-api-access-6mm87") pod "c5d02e26-d0c0-48ed-b8be-ec4287f6986e" (UID: "c5d02e26-d0c0-48ed-b8be-ec4287f6986e"). InnerVolumeSpecName "kube-api-access-6mm87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:00:21.456576 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.456474 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4680c9ea-077e-4e2e-90d6-169088b146a9-kube-api-access-fhgk2" (OuterVolumeSpecName: "kube-api-access-fhgk2") pod "4680c9ea-077e-4e2e-90d6-169088b146a9" (UID: "4680c9ea-077e-4e2e-90d6-169088b146a9"). InnerVolumeSpecName "kube-api-access-fhgk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:00:21.482921 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.482888 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4680c9ea-077e-4e2e-90d6-169088b146a9-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "4680c9ea-077e-4e2e-90d6-169088b146a9" (UID: "4680c9ea-077e-4e2e-90d6-169088b146a9"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:00:21.555800 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.555768 2552 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/4680c9ea-077e-4e2e-90d6-169088b146a9-postgresdata\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:00:21.555800 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.555797 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhgk2\" (UniqueName: \"kubernetes.io/projected/4680c9ea-077e-4e2e-90d6-169088b146a9-kube-api-access-fhgk2\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:00:21.555921 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.555806 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mm87\" (UniqueName: \"kubernetes.io/projected/c5d02e26-d0c0-48ed-b8be-ec4287f6986e-kube-api-access-6mm87\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:00:21.914539 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.914459 2552 generic.go:356] "Generic (PLEG): container finished" podID="c5d02e26-d0c0-48ed-b8be-ec4287f6986e" containerID="8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4" exitCode=0 Sep 15 07:00:21.914539 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.914506 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" Sep 15 07:00:21.914539 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.914533 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" event={"ID":"c5d02e26-d0c0-48ed-b8be-ec4287f6986e","Type":"ContainerDied","Data":"8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4"} Sep 15 07:00:21.915069 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.914567 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-jp56l" event={"ID":"c5d02e26-d0c0-48ed-b8be-ec4287f6986e","Type":"ContainerDied","Data":"67e66e7ff6d5e41e9a47883e6a1d2427c24e96567e44dce2ed9bca4cd625a07a"} Sep 15 07:00:21.915069 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.914588 2552 scope.go:117] "RemoveContainer" containerID="8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4" Sep 15 07:00:21.915862 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.915838 2552 generic.go:356] "Generic (PLEG): container finished" podID="4680c9ea-077e-4e2e-90d6-169088b146a9" containerID="e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff" exitCode=0 Sep 15 07:00:21.915960 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.915876 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" Sep 15 07:00:21.915960 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.915902 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" event={"ID":"4680c9ea-077e-4e2e-90d6-169088b146a9","Type":"ContainerDied","Data":"e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff"} Sep 15 07:00:21.915960 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.915935 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-zqt2t" event={"ID":"4680c9ea-077e-4e2e-90d6-169088b146a9","Type":"ContainerDied","Data":"3812d42d9ef20296c571a669a09c59e3d4a3485f0de296d057addd9eba4cef92"} Sep 15 07:00:21.925335 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.924313 2552 scope.go:117] "RemoveContainer" containerID="8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4" Sep 15 07:00:21.925335 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:00:21.924826 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4\": container with ID starting with 8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4 not found: ID does not exist" containerID="8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4" Sep 15 07:00:21.925335 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.924863 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4"} err="failed to get container status \"8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4\": rpc error: code = NotFound desc = could not find container \"8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4\": container with ID starting with 8124bcd48a7f7895f6fbd7d2dae2c8a21fe9350016a12a70abe2c05a280d91c4 not found: ID does not exist" Sep 15 07:00:21.925335 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.924888 2552 scope.go:117] "RemoveContainer" containerID="e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff" Sep 15 07:00:21.933260 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.933233 2552 scope.go:117] "RemoveContainer" containerID="e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff" Sep 15 07:00:21.933528 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:00:21.933510 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff\": container with ID starting with e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff not found: ID does not exist" containerID="e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff" Sep 15 07:00:21.933568 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.933535 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff"} err="failed to get container status \"e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff\": rpc error: code = NotFound desc = could not find container \"e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff\": container with ID starting with e3d20dfcddfa3c9eb79f8d100755ec39a3e46df8dec85e58b83c35772a3fe5ff not found: ID does not exist" Sep 15 07:00:21.940842 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.940821 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jp56l"] Sep 15 07:00:21.945163 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.945143 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-jp56l"] Sep 15 07:00:21.954771 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.954741 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-zqt2t"] Sep 15 07:00:21.956725 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:21.956703 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-zqt2t"] Sep 15 07:00:22.860692 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:22.860662 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4680c9ea-077e-4e2e-90d6-169088b146a9" path="/var/lib/kubelet/pods/4680c9ea-077e-4e2e-90d6-169088b146a9/volumes" Sep 15 07:00:22.860990 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:22.860961 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5d02e26-d0c0-48ed-b8be-ec4287f6986e" path="/var/lib/kubelet/pods/c5d02e26-d0c0-48ed-b8be-ec4287f6986e/volumes" Sep 15 07:00:30.193365 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193288 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm"] Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193558 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d02e26-d0c0-48ed-b8be-ec4287f6986e" containerName="redis" Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193570 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d02e26-d0c0-48ed-b8be-ec4287f6986e" containerName="redis" Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193588 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39101fe5-602d-4bff-9bf6-01849264a1e8" containerName="manager" Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193593 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="39101fe5-602d-4bff-9bf6-01849264a1e8" containerName="manager" Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193600 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4680c9ea-077e-4e2e-90d6-169088b146a9" containerName="postgres" Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193605 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4680c9ea-077e-4e2e-90d6-169088b146a9" containerName="postgres" Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193649 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="39101fe5-602d-4bff-9bf6-01849264a1e8" containerName="manager" Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193656 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4680c9ea-077e-4e2e-90d6-169088b146a9" containerName="postgres" Sep 15 07:00:30.193742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.193662 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5d02e26-d0c0-48ed-b8be-ec4287f6986e" containerName="redis" Sep 15 07:00:30.200866 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.200844 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:00:30.204287 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.204268 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 15 07:00:30.205592 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.205563 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 15 07:00:30.205717 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.205577 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-mdd4g\"" Sep 15 07:00:30.207019 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.206998 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm"] Sep 15 07:00:30.319747 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.319714 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/2236bb72-97ed-4f31-b54e-4798d17e2d4c-kube-api-access-ng6k8\") pod \"feast-operator-controller-manager-689b877b4b-hx4mm\" (UID: \"2236bb72-97ed-4f31-b54e-4798d17e2d4c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:00:30.420683 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.420650 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/2236bb72-97ed-4f31-b54e-4798d17e2d4c-kube-api-access-ng6k8\") pod \"feast-operator-controller-manager-689b877b4b-hx4mm\" (UID: \"2236bb72-97ed-4f31-b54e-4798d17e2d4c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:00:30.438428 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.438404 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/2236bb72-97ed-4f31-b54e-4798d17e2d4c-kube-api-access-ng6k8\") pod \"feast-operator-controller-manager-689b877b4b-hx4mm\" (UID: \"2236bb72-97ed-4f31-b54e-4798d17e2d4c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:00:30.511438 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.511355 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:00:30.778905 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.778885 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm"] Sep 15 07:00:30.781158 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:00:30.781127 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2236bb72_97ed_4f31_b54e_4798d17e2d4c.slice/crio-d271e2dbded67ac71034467844ec4638558975a7e7ab3fd116ea41c7e7cf4a13 WatchSource:0}: Error finding container d271e2dbded67ac71034467844ec4638558975a7e7ab3fd116ea41c7e7cf4a13: Status 404 returned error can't find the container with id d271e2dbded67ac71034467844ec4638558975a7e7ab3fd116ea41c7e7cf4a13 Sep 15 07:00:30.782931 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.782915 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:00:30.950020 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:30.949985 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" event={"ID":"2236bb72-97ed-4f31-b54e-4798d17e2d4c","Type":"ContainerStarted","Data":"d271e2dbded67ac71034467844ec4638558975a7e7ab3fd116ea41c7e7cf4a13"} Sep 15 07:00:34.967547 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:34.967501 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" event={"ID":"2236bb72-97ed-4f31-b54e-4798d17e2d4c","Type":"ContainerStarted","Data":"5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234"} Sep 15 07:00:34.967926 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:34.967602 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:00:34.984849 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:34.984793 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" podStartSLOduration=1.744876325 podStartE2EDuration="4.984776057s" totalImagesPullingTime="3.239899732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:00:34.983836111 +0000 UTC m=+1438.538493534" watchObservedRunningTime="2026-09-15 07:00:34.984776057 +0000 UTC m=+1438.539433510" Sep 15 07:00:36.890909 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:36.890877 2552 scope.go:117] "RemoveContainer" containerID="a5da4a0e77ba111ec719595ce6f9456a33164da9d400d23240b2148116c75371" Sep 15 07:00:36.898390 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:36.898355 2552 scope.go:117] "RemoveContainer" containerID="6af19b6dbc2fb6d87b988df02ff9663e55521e73d801e5734b9d5558cb09f096" Sep 15 07:00:45.973882 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:45.973850 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:00:47.063932 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.063875 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp"] Sep 15 07:00:47.067334 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.067312 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.069956 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.069931 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 15 07:00:47.070090 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.070045 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 15 07:00:47.070153 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.070113 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 15 07:00:47.070153 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.070140 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-nhq8t\"" Sep 15 07:00:47.070315 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.070141 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 15 07:00:47.070315 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.070214 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 15 07:00:47.070425 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.070415 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 15 07:00:47.081338 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.081318 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp"] Sep 15 07:00:47.149130 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.149094 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5fq4\" (UniqueName: \"kubernetes.io/projected/c31a33ba-e603-4678-9155-b854bed145b8-kube-api-access-z5fq4\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.149130 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.149135 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c31a33ba-e603-4678-9155-b854bed145b8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.149333 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.149206 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.149333 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.149231 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.149333 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.149289 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.149333 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.149331 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.250239 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.250204 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5fq4\" (UniqueName: \"kubernetes.io/projected/c31a33ba-e603-4678-9155-b854bed145b8-kube-api-access-z5fq4\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.250239 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.250240 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c31a33ba-e603-4678-9155-b854bed145b8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.250460 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.250261 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.250460 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.250278 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.250460 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.250296 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.250460 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.250325 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.250460 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:00:47.250428 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 15 07:00:47.250460 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:00:47.250441 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 15 07:00:47.250739 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:00:47.250525 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls podName:c31a33ba-e603-4678-9155-b854bed145b8 nodeName:}" failed. No retries permitted until 2026-09-15 07:00:47.750505121 +0000 UTC m=+1451.305162522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-9cdhp" (UID: "c31a33ba-e603-4678-9155-b854bed145b8") : secret "feast-simple-feast-setup-ui-tls" not found Sep 15 07:00:47.250739 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:00:47.250542 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls podName:c31a33ba-e603-4678-9155-b854bed145b8 nodeName:}" failed. No retries permitted until 2026-09-15 07:00:47.750534301 +0000 UTC m=+1451.305191702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls") pod "feast-simple-feast-setup-57576b8fcf-9cdhp" (UID: "c31a33ba-e603-4678-9155-b854bed145b8") : secret "feast-simple-feast-setup-online-tls" not found Sep 15 07:00:47.250739 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.250696 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c31a33ba-e603-4678-9155-b854bed145b8-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.252688 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.252663 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.252837 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.252818 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.259004 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.258982 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5fq4\" (UniqueName: \"kubernetes.io/projected/c31a33ba-e603-4678-9155-b854bed145b8-kube-api-access-z5fq4\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.753906 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.753872 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.754113 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.753936 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.756365 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.756341 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.756455 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.756432 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-9cdhp\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:47.976239 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:47.976194 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:00:48.246764 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:48.246737 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp"] Sep 15 07:00:48.247706 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:00:48.247687 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc31a33ba_e603_4678_9155_b854bed145b8.slice/crio-16c3710d6dd01e7d1bd1c8c7eaa302fce3c4466e90426490bdfe88f464ff2d84 WatchSource:0}: Error finding container 16c3710d6dd01e7d1bd1c8c7eaa302fce3c4466e90426490bdfe88f464ff2d84: Status 404 returned error can't find the container with id 16c3710d6dd01e7d1bd1c8c7eaa302fce3c4466e90426490bdfe88f464ff2d84 Sep 15 07:00:49.019437 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:00:49.019398 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerStarted","Data":"16c3710d6dd01e7d1bd1c8c7eaa302fce3c4466e90426490bdfe88f464ff2d84"} Sep 15 07:01:04.082199 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:04.082160 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerStarted","Data":"0286c0f3678d95c7029b96c9351fe202ec4f83db9555789da91792cd64ac766d"} Sep 15 07:01:09.102476 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:09.102416 2552 generic.go:356] "Generic (PLEG): container finished" podID="c31a33ba-e603-4678-9155-b854bed145b8" containerID="0286c0f3678d95c7029b96c9351fe202ec4f83db9555789da91792cd64ac766d" exitCode=0 Sep 15 07:01:09.102935 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:09.102491 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerDied","Data":"0286c0f3678d95c7029b96c9351fe202ec4f83db9555789da91792cd64ac766d"} Sep 15 07:01:10.108889 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:10.108847 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerStarted","Data":"4ab0ed5bdf32397b1960d9abebb4c5cf6b0104d9fa6005a5d088f600acf38389"} Sep 15 07:01:15.129513 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:15.129481 2552 generic.go:356] "Generic (PLEG): container finished" podID="c31a33ba-e603-4678-9155-b854bed145b8" containerID="4ab0ed5bdf32397b1960d9abebb4c5cf6b0104d9fa6005a5d088f600acf38389" exitCode=0 Sep 15 07:01:15.129884 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:15.129531 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerDied","Data":"4ab0ed5bdf32397b1960d9abebb4c5cf6b0104d9fa6005a5d088f600acf38389"} Sep 15 07:01:16.139757 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:16.139708 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerStarted","Data":"39e90e113fbc707c213215b4794968be9ce7014cc3a3f32b507e0d1f91b72501"} Sep 15 07:01:16.140259 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:16.139775 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerStarted","Data":"7dfe812ec359337ebff3c1077b281f728d33207ab5e314d11bf68493b655919f"} Sep 15 07:01:16.140259 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:16.139797 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerStarted","Data":"5a969c450da9ecc340b8324059149a869293071a1d565478dd5392c60bac8370"} Sep 15 07:01:16.140259 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:16.139815 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerStarted","Data":"7f98541116219d5a9939a228d14f0b59fdfa61e607d43e40ecf7cf4d8fcf01d3"} Sep 15 07:01:16.162760 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:16.162699 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podStartSLOduration=14.246366857 podStartE2EDuration="29.16267587s" totalImagesPullingTime="14.916309013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:01:16.160980468 +0000 UTC m=+1479.715637907" watchObservedRunningTime="2026-09-15 07:01:16.16267587 +0000 UTC m=+1479.717333293" Sep 15 07:01:17.976718 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.976675 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:17.977298 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.977274 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:17.977431 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.977308 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:17.977431 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.977327 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:17.977431 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.977343 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:17.977431 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.977359 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:17.977431 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.977374 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:17.977431 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.977391 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:17.979404 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.979253 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 15 07:01:17.979404 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.979296 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Sep 15 07:01:17.979404 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.979336 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 15 07:01:17.979404 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:17.979295 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 15 07:01:20.977525 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:20.977474 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Sep 15 07:01:20.978054 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:20.977475 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 15 07:01:20.978054 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:20.977550 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Sep 15 07:01:20.978054 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:20.977651 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 15 07:01:23.977491 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:23.977349 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:23.978218 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:23.978195 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:23.978638 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:23.978615 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:23.978830 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:23.978809 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:23.984560 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:23.984543 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:24.183157 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:24.183128 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:24.183500 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:24.183483 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:01:24.186030 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:01:24.186007 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:02:25.377943 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.377751 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp"] Sep 15 07:02:25.378663 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.378584 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="registry" containerID="cri-o://7f98541116219d5a9939a228d14f0b59fdfa61e607d43e40ecf7cf4d8fcf01d3" gracePeriod=30 Sep 15 07:02:25.378868 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.378619 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" containerID="cri-o://39e90e113fbc707c213215b4794968be9ce7014cc3a3f32b507e0d1f91b72501" gracePeriod=30 Sep 15 07:02:25.379020 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.378654 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" containerID="cri-o://5a969c450da9ecc340b8324059149a869293071a1d565478dd5392c60bac8370" gracePeriod=30 Sep 15 07:02:25.379020 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.378651 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="offline" containerID="cri-o://7dfe812ec359337ebff3c1077b281f728d33207ab5e314d11bf68493b655919f" gracePeriod=30 Sep 15 07:02:25.790267 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.790226 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx"] Sep 15 07:02:25.794233 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.794210 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:25.796896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.796871 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-gxwbf\"" Sep 15 07:02:25.805433 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.805407 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx"] Sep 15 07:02:25.920356 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.920317 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:25.920543 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.920398 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:25.920543 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.920436 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be565ff7-465e-438c-bcb0-c13c7dbef853-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:25.920543 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.920494 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxsk7\" (UniqueName: \"kubernetes.io/projected/be565ff7-465e-438c-bcb0-c13c7dbef853-kube-api-access-fxsk7\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:25.920691 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.920613 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:25.920691 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:25.920658 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.021346 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.021306 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be565ff7-465e-438c-bcb0-c13c7dbef853-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.021516 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.021354 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsk7\" (UniqueName: \"kubernetes.io/projected/be565ff7-465e-438c-bcb0-c13c7dbef853-kube-api-access-fxsk7\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.021516 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.021385 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.021516 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:02:26.021477 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 15 07:02:26.021633 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:02:26.021555 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls podName:be565ff7-465e-438c-bcb0-c13c7dbef853 nodeName:}" failed. No retries permitted until 2026-09-15 07:02:26.521534136 +0000 UTC m=+1550.076191558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-d49qx" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853") : secret "feast-simple-feast-setup-registry-tls" not found Sep 15 07:02:26.021633 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.021574 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.021716 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.021641 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.021716 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.021705 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.021794 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.021748 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be565ff7-465e-438c-bcb0-c13c7dbef853-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.021794 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:02:26.021782 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 15 07:02:26.021891 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:02:26.021809 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 15 07:02:26.021891 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:02:26.021834 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls podName:be565ff7-465e-438c-bcb0-c13c7dbef853 nodeName:}" failed. No retries permitted until 2026-09-15 07:02:26.521820706 +0000 UTC m=+1550.076478108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls") pod "feast-simple-feast-setup-57576b8fcf-d49qx" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853") : secret "feast-simple-feast-setup-online-tls" not found Sep 15 07:02:26.021891 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:02:26.021856 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls podName:be565ff7-465e-438c-bcb0-c13c7dbef853 nodeName:}" failed. No retries permitted until 2026-09-15 07:02:26.521844774 +0000 UTC m=+1550.076502175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-d49qx" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853") : secret "feast-simple-feast-setup-offline-tls" not found Sep 15 07:02:26.024309 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.024282 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.029594 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.029563 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxsk7\" (UniqueName: \"kubernetes.io/projected/be565ff7-465e-438c-bcb0-c13c7dbef853-kube-api-access-fxsk7\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.410912 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.410833 2552 generic.go:356] "Generic (PLEG): container finished" podID="c31a33ba-e603-4678-9155-b854bed145b8" containerID="39e90e113fbc707c213215b4794968be9ce7014cc3a3f32b507e0d1f91b72501" exitCode=0 Sep 15 07:02:26.410912 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.410859 2552 generic.go:356] "Generic (PLEG): container finished" podID="c31a33ba-e603-4678-9155-b854bed145b8" containerID="5a969c450da9ecc340b8324059149a869293071a1d565478dd5392c60bac8370" exitCode=0 Sep 15 07:02:26.411320 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.410903 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerDied","Data":"39e90e113fbc707c213215b4794968be9ce7014cc3a3f32b507e0d1f91b72501"} Sep 15 07:02:26.411320 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.410952 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerDied","Data":"5a969c450da9ecc340b8324059149a869293071a1d565478dd5392c60bac8370"} Sep 15 07:02:26.525559 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.525528 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.525691 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.525577 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.525691 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.525613 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.525691 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:02:26.525689 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 15 07:02:26.525794 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:02:26.525749 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls podName:be565ff7-465e-438c-bcb0-c13c7dbef853 nodeName:}" failed. No retries permitted until 2026-09-15 07:02:27.525735185 +0000 UTC m=+1551.080392587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-d49qx" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853") : secret "feast-simple-feast-setup-registry-tls" not found Sep 15 07:02:26.527906 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.527880 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:26.528050 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:26.528031 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:27.534747 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:27.534711 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:27.537063 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:27.537044 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-d49qx\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:27.605360 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:27.605331 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:27.878035 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:27.878004 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx"] Sep 15 07:02:27.879544 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:02:27.879518 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe565ff7_465e_438c_bcb0_c13c7dbef853.slice/crio-33ef02502ff6012290f5365e4e5994234c6688d7dcd0e356020b7b34ec97aa07 WatchSource:0}: Error finding container 33ef02502ff6012290f5365e4e5994234c6688d7dcd0e356020b7b34ec97aa07: Status 404 returned error can't find the container with id 33ef02502ff6012290f5365e4e5994234c6688d7dcd0e356020b7b34ec97aa07 Sep 15 07:02:28.420317 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:28.420284 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerStarted","Data":"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942"} Sep 15 07:02:28.420492 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:28.420326 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerStarted","Data":"33ef02502ff6012290f5365e4e5994234c6688d7dcd0e356020b7b34ec97aa07"} Sep 15 07:02:33.441823 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:33.441792 2552 generic.go:356] "Generic (PLEG): container finished" podID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerID="c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942" exitCode=0 Sep 15 07:02:33.442215 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:33.441830 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerDied","Data":"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942"} Sep 15 07:02:33.978216 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:33.978168 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 15 07:02:34.183561 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:34.183505 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 15 07:02:34.448121 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:34.448081 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerStarted","Data":"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e"} Sep 15 07:02:39.469732 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:39.469651 2552 generic.go:356] "Generic (PLEG): container finished" podID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerID="b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e" exitCode=0 Sep 15 07:02:39.469732 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:39.469723 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerDied","Data":"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e"} Sep 15 07:02:40.479920 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:40.479875 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerStarted","Data":"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b"} Sep 15 07:02:40.480419 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:40.479939 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerStarted","Data":"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086"} Sep 15 07:02:40.480419 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:40.479960 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerStarted","Data":"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c"} Sep 15 07:02:40.480419 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:40.479998 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerStarted","Data":"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e"} Sep 15 07:02:40.505047 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:40.504983 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podStartSLOduration=15.504944283 podStartE2EDuration="15.504944283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:02:40.502800104 +0000 UTC m=+1564.057457555" watchObservedRunningTime="2026-09-15 07:02:40.504944283 +0000 UTC m=+1564.059601706" Sep 15 07:02:42.605894 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:42.605857 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:42.606460 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:42.606440 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:42.606583 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:42.606571 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:42.607014 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:42.606997 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:42.608219 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:42.608177 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 15 07:02:42.608326 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:42.608261 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 15 07:02:42.608326 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:42.608270 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 15 07:02:42.608326 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:42.608283 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:02:43.978951 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:43.978904 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 15 07:02:44.183644 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:44.183252 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 15 07:02:45.606286 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:45.606239 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.27:6571: connect: connection refused" Sep 15 07:02:45.606874 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:45.606239 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 15 07:02:45.607023 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:45.606254 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Sep 15 07:02:45.607023 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:45.606408 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:02:47.605453 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:47.605420 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:47.605866 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:47.605482 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:47.605866 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:47.605495 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:47.605866 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:47.605506 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:48.606803 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:48.606772 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:48.609278 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:48.606888 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:48.609278 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:48.607110 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:48.609278 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:48.607707 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:48.609278 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:48.607764 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:48.609278 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:48.607901 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:48.611913 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:48.611898 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:49.528825 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:49.528797 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:02:53.978106 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:53.978070 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Sep 15 07:02:53.978463 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:53.978181 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:02:54.182947 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:54.182893 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Sep 15 07:02:54.183249 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:54.183068 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:02:55.549318 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:55.549276 2552 generic.go:356] "Generic (PLEG): container finished" podID="c31a33ba-e603-4678-9155-b854bed145b8" containerID="7dfe812ec359337ebff3c1077b281f728d33207ab5e314d11bf68493b655919f" exitCode=137 Sep 15 07:02:55.549318 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:55.549305 2552 generic.go:356] "Generic (PLEG): container finished" podID="c31a33ba-e603-4678-9155-b854bed145b8" containerID="7f98541116219d5a9939a228d14f0b59fdfa61e607d43e40ecf7cf4d8fcf01d3" exitCode=137 Sep 15 07:02:55.549709 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:55.549342 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerDied","Data":"7dfe812ec359337ebff3c1077b281f728d33207ab5e314d11bf68493b655919f"} Sep 15 07:02:55.549709 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:55.549383 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerDied","Data":"7f98541116219d5a9939a228d14f0b59fdfa61e607d43e40ecf7cf4d8fcf01d3"} Sep 15 07:02:56.017573 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.017549 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:02:56.087127 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.087097 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls\") pod \"c31a33ba-e603-4678-9155-b854bed145b8\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " Sep 15 07:02:56.087297 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.087134 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c31a33ba-e603-4678-9155-b854bed145b8-feast-data\") pod \"c31a33ba-e603-4678-9155-b854bed145b8\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " Sep 15 07:02:56.087297 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.087176 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-registry-tls\") pod \"c31a33ba-e603-4678-9155-b854bed145b8\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " Sep 15 07:02:56.087297 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.087198 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5fq4\" (UniqueName: \"kubernetes.io/projected/c31a33ba-e603-4678-9155-b854bed145b8-kube-api-access-z5fq4\") pod \"c31a33ba-e603-4678-9155-b854bed145b8\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " Sep 15 07:02:56.087297 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.087215 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls\") pod \"c31a33ba-e603-4678-9155-b854bed145b8\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " Sep 15 07:02:56.087297 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.087241 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-offline-tls\") pod \"c31a33ba-e603-4678-9155-b854bed145b8\" (UID: \"c31a33ba-e603-4678-9155-b854bed145b8\") " Sep 15 07:02:56.087877 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.087834 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31a33ba-e603-4678-9155-b854bed145b8-feast-data" (OuterVolumeSpecName: "feast-data") pod "c31a33ba-e603-4678-9155-b854bed145b8" (UID: "c31a33ba-e603-4678-9155-b854bed145b8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:02:56.089571 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.089549 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c31a33ba-e603-4678-9155-b854bed145b8" (UID: "c31a33ba-e603-4678-9155-b854bed145b8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:02:56.089833 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.089813 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c31a33ba-e603-4678-9155-b854bed145b8" (UID: "c31a33ba-e603-4678-9155-b854bed145b8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:02:56.089911 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.089833 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls" (OuterVolumeSpecName: "online-tls") pod "c31a33ba-e603-4678-9155-b854bed145b8" (UID: "c31a33ba-e603-4678-9155-b854bed145b8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:02:56.089911 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.089858 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c31a33ba-e603-4678-9155-b854bed145b8" (UID: "c31a33ba-e603-4678-9155-b854bed145b8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:02:56.089911 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.089867 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c31a33ba-e603-4678-9155-b854bed145b8-kube-api-access-z5fq4" (OuterVolumeSpecName: "kube-api-access-z5fq4") pod "c31a33ba-e603-4678-9155-b854bed145b8" (UID: "c31a33ba-e603-4678-9155-b854bed145b8"). InnerVolumeSpecName "kube-api-access-z5fq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:02:56.188327 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.188302 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-online-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:02:56.188327 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.188327 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c31a33ba-e603-4678-9155-b854bed145b8-feast-data\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:02:56.188498 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.188337 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-registry-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:02:56.188498 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.188346 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5fq4\" (UniqueName: \"kubernetes.io/projected/c31a33ba-e603-4678-9155-b854bed145b8-kube-api-access-z5fq4\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:02:56.188498 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.188355 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-ui-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:02:56.188498 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.188364 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c31a33ba-e603-4678-9155-b854bed145b8-offline-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:02:56.556723 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.556641 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" Sep 15 07:02:56.557290 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.556641 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp" event={"ID":"c31a33ba-e603-4678-9155-b854bed145b8","Type":"ContainerDied","Data":"16c3710d6dd01e7d1bd1c8c7eaa302fce3c4466e90426490bdfe88f464ff2d84"} Sep 15 07:02:56.557290 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.556767 2552 scope.go:117] "RemoveContainer" containerID="39e90e113fbc707c213215b4794968be9ce7014cc3a3f32b507e0d1f91b72501" Sep 15 07:02:56.565596 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.565576 2552 scope.go:117] "RemoveContainer" containerID="7dfe812ec359337ebff3c1077b281f728d33207ab5e314d11bf68493b655919f" Sep 15 07:02:56.572932 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.572916 2552 scope.go:117] "RemoveContainer" containerID="5a969c450da9ecc340b8324059149a869293071a1d565478dd5392c60bac8370" Sep 15 07:02:56.580617 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.580596 2552 scope.go:117] "RemoveContainer" containerID="7f98541116219d5a9939a228d14f0b59fdfa61e607d43e40ecf7cf4d8fcf01d3" Sep 15 07:02:56.580617 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.580615 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp"] Sep 15 07:02:56.585738 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.585709 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-9cdhp"] Sep 15 07:02:56.588251 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.588232 2552 scope.go:117] "RemoveContainer" containerID="4ab0ed5bdf32397b1960d9abebb4c5cf6b0104d9fa6005a5d088f600acf38389" Sep 15 07:02:56.595068 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.595052 2552 scope.go:117] "RemoveContainer" containerID="0286c0f3678d95c7029b96c9351fe202ec4f83db9555789da91792cd64ac766d" Sep 15 07:02:56.862070 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:02:56.861995 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c31a33ba-e603-4678-9155-b854bed145b8" path="/var/lib/kubelet/pods/c31a33ba-e603-4678-9155-b854bed145b8/volumes" Sep 15 07:03:52.782470 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782437 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79"] Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782719 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782730 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782742 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="offline" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782747 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="offline" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782759 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="registry" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782765 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="registry" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782773 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782780 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782789 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="feast-init" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782794 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="feast-init" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782800 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="feast-apply" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782805 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="feast-apply" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782849 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="registry" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782857 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="offline" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782863 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="ui" Sep 15 07:03:52.782896 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.782868 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c31a33ba-e603-4678-9155-b854bed145b8" containerName="online" Sep 15 07:03:52.786028 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.786007 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:52.788684 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.788625 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 15 07:03:52.789285 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.789256 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 15 07:03:52.789407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.789303 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 15 07:03:52.789407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.789338 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-7p8gx\"" Sep 15 07:03:52.789407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.789387 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 15 07:03:52.789861 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.789842 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 15 07:03:52.789929 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.789872 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 15 07:03:52.796885 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.796861 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79"] Sep 15 07:03:52.922784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.922750 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a36c53a7-fafb-45e4-b803-7b14d983d665-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:52.922784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.922790 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kc7n\" (UniqueName: \"kubernetes.io/projected/a36c53a7-fafb-45e4-b803-7b14d983d665-kube-api-access-9kc7n\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:52.923029 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.922809 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:52.923029 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.922899 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a36c53a7-fafb-45e4-b803-7b14d983d665-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:52.923029 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.922915 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:52.923029 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:52.922932 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.023839 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.023807 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a36c53a7-fafb-45e4-b803-7b14d983d665-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.023839 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.023841 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.024109 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.023868 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.024109 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.023955 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a36c53a7-fafb-45e4-b803-7b14d983d665-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.024109 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.024011 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kc7n\" (UniqueName: \"kubernetes.io/projected/a36c53a7-fafb-45e4-b803-7b14d983d665-kube-api-access-9kc7n\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.024109 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.024043 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.024304 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:03:53.024151 2552 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 15 07:03:53.024304 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:03:53.024235 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls podName:a36c53a7-fafb-45e4-b803-7b14d983d665 nodeName:}" failed. No retries permitted until 2026-09-15 07:03:53.524214083 +0000 UTC m=+1637.078871492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-pll79" (UID: "a36c53a7-fafb-45e4-b803-7b14d983d665") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 15 07:03:53.024470 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.024443 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a36c53a7-fafb-45e4-b803-7b14d983d665-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.024735 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.024715 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a36c53a7-fafb-45e4-b803-7b14d983d665-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.026467 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.026442 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.026569 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.026477 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.032749 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.032683 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kc7n\" (UniqueName: \"kubernetes.io/projected/a36c53a7-fafb-45e4-b803-7b14d983d665-kube-api-access-9kc7n\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.529141 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.529103 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.531539 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.531510 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-pll79\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.701019 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.700947 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:03:53.972103 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:53.972076 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79"] Sep 15 07:03:53.973185 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:03:53.973160 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda36c53a7_fafb_45e4_b803_7b14d983d665.slice/crio-197d7bf3910def34b06dde81e589c2c795f4fe8cb19f1be5c7edd1482bec00cc WatchSource:0}: Error finding container 197d7bf3910def34b06dde81e589c2c795f4fe8cb19f1be5c7edd1482bec00cc: Status 404 returned error can't find the container with id 197d7bf3910def34b06dde81e589c2c795f4fe8cb19f1be5c7edd1482bec00cc Sep 15 07:03:54.776525 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:54.776489 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerStarted","Data":"59449698d5a6384c07e8490b603cc3d138094c1698ed4a25e90dd55e90ba0f20"} Sep 15 07:03:54.776687 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:54.776534 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerStarted","Data":"197d7bf3910def34b06dde81e589c2c795f4fe8cb19f1be5c7edd1482bec00cc"} Sep 15 07:03:59.800281 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:59.800246 2552 generic.go:356] "Generic (PLEG): container finished" podID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerID="59449698d5a6384c07e8490b603cc3d138094c1698ed4a25e90dd55e90ba0f20" exitCode=0 Sep 15 07:03:59.800281 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:03:59.800286 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerDied","Data":"59449698d5a6384c07e8490b603cc3d138094c1698ed4a25e90dd55e90ba0f20"} Sep 15 07:04:00.806141 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:00.806108 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerStarted","Data":"fb770377e99c315708d284bd1764503434b4750332e64cb4e53b40828d9cc45f"} Sep 15 07:04:22.891900 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:22.891866 2552 generic.go:356] "Generic (PLEG): container finished" podID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerID="fb770377e99c315708d284bd1764503434b4750332e64cb4e53b40828d9cc45f" exitCode=0 Sep 15 07:04:22.892288 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:22.891934 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerDied","Data":"fb770377e99c315708d284bd1764503434b4750332e64cb4e53b40828d9cc45f"} Sep 15 07:04:23.900538 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:23.900500 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerStarted","Data":"1721ae811e1df0c0083f9e3881c126862584cda7cbbbf21bb196e92286608995"} Sep 15 07:04:23.901041 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:23.900557 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerStarted","Data":"056910b482298820c063582c9e526f7d511f8c3e9f644d410c12b892dd069a37"} Sep 15 07:04:23.901041 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:23.900571 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerStarted","Data":"06f419db33738ab946a3f9cc24f9126b747160a133e4c600d60ea6ee76d7e5a1"} Sep 15 07:04:23.926184 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:23.926123 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podStartSLOduration=31.926102015 podStartE2EDuration="31.926102015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:03:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:04:23.924833839 +0000 UTC m=+1667.479491261" watchObservedRunningTime="2026-09-15 07:04:23.926102015 +0000 UTC m=+1667.480759439" Sep 15 07:04:26.701255 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:26.701203 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:26.701738 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:26.701285 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:26.701738 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:26.701303 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:26.703014 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:26.702868 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused" Sep 15 07:04:26.703014 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:26.702896 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 15 07:04:26.703014 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:26.702900 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:04:29.701561 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:29.701512 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 15 07:04:29.702120 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:29.701599 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:04:29.702120 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:29.701526 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused" Sep 15 07:04:32.702508 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:32.702474 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:32.703060 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:32.703038 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:32.703125 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:32.703082 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:32.703356 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:32.703333 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:32.940817 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:32.940781 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:32.941115 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:32.941095 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:33.701514 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:33.701478 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:33.705080 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:33.705053 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 15 07:04:36.471774 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:36.471739 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:04:37.257891 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:04:37.257857 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:05:34.706196 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:34.706092 2552 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 15 07:05:39.210522 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.210487 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79"] Sep 15 07:05:39.211002 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.210714 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" containerID="cri-o://06f419db33738ab946a3f9cc24f9126b747160a133e4c600d60ea6ee76d7e5a1" gracePeriod=30 Sep 15 07:05:39.211002 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.210771 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="offline" containerID="cri-o://056910b482298820c063582c9e526f7d511f8c3e9f644d410c12b892dd069a37" gracePeriod=30 Sep 15 07:05:39.211002 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.210769 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" containerID="cri-o://1721ae811e1df0c0083f9e3881c126862584cda7cbbbf21bb196e92286608995" gracePeriod=30 Sep 15 07:05:39.356249 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.356141 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx"] Sep 15 07:05:39.356508 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.356442 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="registry" containerID="cri-o://8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e" gracePeriod=30 Sep 15 07:05:39.356508 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.356488 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="offline" containerID="cri-o://2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086" gracePeriod=30 Sep 15 07:05:39.356718 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.356501 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" containerID="cri-o://974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c" gracePeriod=30 Sep 15 07:05:39.356718 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.356530 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" containerID="cri-o://a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b" gracePeriod=30 Sep 15 07:05:39.526604 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:39.526493 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:05:40.208679 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:40.208645 2552 generic.go:356] "Generic (PLEG): container finished" podID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerID="a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b" exitCode=0 Sep 15 07:05:40.208855 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:40.208712 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerDied","Data":"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b"} Sep 15 07:05:40.211333 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:40.211311 2552 generic.go:356] "Generic (PLEG): container finished" podID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerID="1721ae811e1df0c0083f9e3881c126862584cda7cbbbf21bb196e92286608995" exitCode=0 Sep 15 07:05:40.211333 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:40.211331 2552 generic.go:356] "Generic (PLEG): container finished" podID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerID="06f419db33738ab946a3f9cc24f9126b747160a133e4c600d60ea6ee76d7e5a1" exitCode=0 Sep 15 07:05:40.211680 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:40.211385 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerDied","Data":"1721ae811e1df0c0083f9e3881c126862584cda7cbbbf21bb196e92286608995"} Sep 15 07:05:40.211680 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:40.211420 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerDied","Data":"06f419db33738ab946a3f9cc24f9126b747160a133e4c600d60ea6ee76d7e5a1"} Sep 15 07:05:41.218111 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:41.218069 2552 generic.go:356] "Generic (PLEG): container finished" podID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerID="974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c" exitCode=0 Sep 15 07:05:41.218476 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:41.218135 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerDied","Data":"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c"} Sep 15 07:05:42.941286 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:42.941241 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 15 07:05:46.472865 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:46.472818 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:05:48.608542 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:48.608487 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 15 07:05:49.526184 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:49.526133 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:05:52.941905 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:52.941865 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 15 07:05:56.472725 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:56.472682 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:05:58.608023 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:58.607951 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 15 07:05:59.526151 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:59.526110 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:05:59.526338 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:05:59.526238 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:06:02.941198 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:02.941149 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 15 07:06:02.941583 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:02.941295 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:06:06.472414 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:06.472361 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 15 07:06:06.473017 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:06.472557 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:06:08.607852 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:08.607801 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Sep 15 07:06:08.608300 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:08.607938 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:06:09.335224 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.335191 2552 generic.go:356] "Generic (PLEG): container finished" podID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerID="056910b482298820c063582c9e526f7d511f8c3e9f644d410c12b892dd069a37" exitCode=137 Sep 15 07:06:09.335395 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.335259 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerDied","Data":"056910b482298820c063582c9e526f7d511f8c3e9f644d410c12b892dd069a37"} Sep 15 07:06:09.526893 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.526792 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Sep 15 07:06:09.853425 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.853399 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:06:09.950511 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.950481 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kc7n\" (UniqueName: \"kubernetes.io/projected/a36c53a7-fafb-45e4-b803-7b14d983d665-kube-api-access-9kc7n\") pod \"a36c53a7-fafb-45e4-b803-7b14d983d665\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " Sep 15 07:06:09.950682 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.950522 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a36c53a7-fafb-45e4-b803-7b14d983d665-registry-tls\") pod \"a36c53a7-fafb-45e4-b803-7b14d983d665\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " Sep 15 07:06:09.950682 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.950543 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-offline-tls\") pod \"a36c53a7-fafb-45e4-b803-7b14d983d665\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " Sep 15 07:06:09.950682 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.950566 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a36c53a7-fafb-45e4-b803-7b14d983d665-feast-data\") pod \"a36c53a7-fafb-45e4-b803-7b14d983d665\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " Sep 15 07:06:09.950682 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.950657 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls\") pod \"a36c53a7-fafb-45e4-b803-7b14d983d665\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " Sep 15 07:06:09.950879 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.950697 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-ui-tls\") pod \"a36c53a7-fafb-45e4-b803-7b14d983d665\" (UID: \"a36c53a7-fafb-45e4-b803-7b14d983d665\") " Sep 15 07:06:09.951205 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.950946 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a36c53a7-fafb-45e4-b803-7b14d983d665-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a36c53a7-fafb-45e4-b803-7b14d983d665" (UID: "a36c53a7-fafb-45e4-b803-7b14d983d665"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:06:09.951374 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.951342 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a36c53a7-fafb-45e4-b803-7b14d983d665-feast-data" (OuterVolumeSpecName: "feast-data") pod "a36c53a7-fafb-45e4-b803-7b14d983d665" (UID: "a36c53a7-fafb-45e4-b803-7b14d983d665"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:06:09.953018 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.952962 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a36c53a7-fafb-45e4-b803-7b14d983d665-kube-api-access-9kc7n" (OuterVolumeSpecName: "kube-api-access-9kc7n") pod "a36c53a7-fafb-45e4-b803-7b14d983d665" (UID: "a36c53a7-fafb-45e4-b803-7b14d983d665"). InnerVolumeSpecName "kube-api-access-9kc7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:06:09.953107 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.953083 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a36c53a7-fafb-45e4-b803-7b14d983d665" (UID: "a36c53a7-fafb-45e4-b803-7b14d983d665"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:06:09.953160 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.953103 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a36c53a7-fafb-45e4-b803-7b14d983d665" (UID: "a36c53a7-fafb-45e4-b803-7b14d983d665"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:06:09.953484 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.953462 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls" (OuterVolumeSpecName: "online-tls") pod "a36c53a7-fafb-45e4-b803-7b14d983d665" (UID: "a36c53a7-fafb-45e4-b803-7b14d983d665"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:06:09.997714 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:09.997687 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:06:10.051915 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.051876 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls\") pod \"be565ff7-465e-438c-bcb0-c13c7dbef853\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " Sep 15 07:06:10.051915 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.051922 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be565ff7-465e-438c-bcb0-c13c7dbef853-feast-data\") pod \"be565ff7-465e-438c-bcb0-c13c7dbef853\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " Sep 15 07:06:10.052171 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.051946 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-ui-tls\") pod \"be565ff7-465e-438c-bcb0-c13c7dbef853\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " Sep 15 07:06:10.052171 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.051997 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls\") pod \"be565ff7-465e-438c-bcb0-c13c7dbef853\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " Sep 15 07:06:10.052171 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052034 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxsk7\" (UniqueName: \"kubernetes.io/projected/be565ff7-465e-438c-bcb0-c13c7dbef853-kube-api-access-fxsk7\") pod \"be565ff7-465e-438c-bcb0-c13c7dbef853\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " Sep 15 07:06:10.052171 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052066 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls\") pod \"be565ff7-465e-438c-bcb0-c13c7dbef853\" (UID: \"be565ff7-465e-438c-bcb0-c13c7dbef853\") " Sep 15 07:06:10.052377 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052271 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a36c53a7-fafb-45e4-b803-7b14d983d665-registry-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.052377 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052290 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-offline-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.052377 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052302 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a36c53a7-fafb-45e4-b803-7b14d983d665-feast-data\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.052377 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052315 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-online-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.052377 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052327 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a36c53a7-fafb-45e4-b803-7b14d983d665-ui-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.052377 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052339 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kc7n\" (UniqueName: \"kubernetes.io/projected/a36c53a7-fafb-45e4-b803-7b14d983d665-kube-api-access-9kc7n\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.052673 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.052536 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be565ff7-465e-438c-bcb0-c13c7dbef853-feast-data" (OuterVolumeSpecName: "feast-data") pod "be565ff7-465e-438c-bcb0-c13c7dbef853" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:06:10.054306 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.054270 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "be565ff7-465e-438c-bcb0-c13c7dbef853" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:06:10.054306 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.054286 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "be565ff7-465e-438c-bcb0-c13c7dbef853" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:06:10.054306 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.054298 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "be565ff7-465e-438c-bcb0-c13c7dbef853" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:06:10.054492 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.054477 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls" (OuterVolumeSpecName: "online-tls") pod "be565ff7-465e-438c-bcb0-c13c7dbef853" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:06:10.054537 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.054490 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be565ff7-465e-438c-bcb0-c13c7dbef853-kube-api-access-fxsk7" (OuterVolumeSpecName: "kube-api-access-fxsk7") pod "be565ff7-465e-438c-bcb0-c13c7dbef853" (UID: "be565ff7-465e-438c-bcb0-c13c7dbef853"). InnerVolumeSpecName "kube-api-access-fxsk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:06:10.153407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.153317 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-online-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.153407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.153348 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/be565ff7-465e-438c-bcb0-c13c7dbef853-feast-data\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.153407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.153357 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-ui-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.153407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.153364 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-registry-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.153407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.153373 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxsk7\" (UniqueName: \"kubernetes.io/projected/be565ff7-465e-438c-bcb0-c13c7dbef853-kube-api-access-fxsk7\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.153407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.153382 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/be565ff7-465e-438c-bcb0-c13c7dbef853-offline-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:10.342167 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.342127 2552 generic.go:356] "Generic (PLEG): container finished" podID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerID="2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086" exitCode=137 Sep 15 07:06:10.342167 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.342156 2552 generic.go:356] "Generic (PLEG): container finished" podID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerID="8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e" exitCode=137 Sep 15 07:06:10.342412 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.342188 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" Sep 15 07:06:10.342412 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.342200 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerDied","Data":"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086"} Sep 15 07:06:10.342412 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.342242 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerDied","Data":"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e"} Sep 15 07:06:10.342412 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.342259 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx" event={"ID":"be565ff7-465e-438c-bcb0-c13c7dbef853","Type":"ContainerDied","Data":"33ef02502ff6012290f5365e4e5994234c6688d7dcd0e356020b7b34ec97aa07"} Sep 15 07:06:10.342412 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.342281 2552 scope.go:117] "RemoveContainer" containerID="a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b" Sep 15 07:06:10.345390 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.345369 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" Sep 15 07:06:10.345509 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.345388 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79" event={"ID":"a36c53a7-fafb-45e4-b803-7b14d983d665","Type":"ContainerDied","Data":"197d7bf3910def34b06dde81e589c2c795f4fe8cb19f1be5c7edd1482bec00cc"} Sep 15 07:06:10.351242 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.351219 2552 scope.go:117] "RemoveContainer" containerID="2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086" Sep 15 07:06:10.359747 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.359728 2552 scope.go:117] "RemoveContainer" containerID="974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c" Sep 15 07:06:10.367492 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.367464 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx"] Sep 15 07:06:10.368153 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.368136 2552 scope.go:117] "RemoveContainer" containerID="8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e" Sep 15 07:06:10.370930 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.370907 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-d49qx"] Sep 15 07:06:10.378819 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.378792 2552 scope.go:117] "RemoveContainer" containerID="b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e" Sep 15 07:06:10.383684 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.383657 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79"] Sep 15 07:06:10.387044 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.387021 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-pll79"] Sep 15 07:06:10.387414 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.387398 2552 scope.go:117] "RemoveContainer" containerID="c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942" Sep 15 07:06:10.395442 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.395424 2552 scope.go:117] "RemoveContainer" containerID="a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b" Sep 15 07:06:10.395737 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:10.395720 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b\": container with ID starting with a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b not found: ID does not exist" containerID="a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b" Sep 15 07:06:10.395799 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.395746 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b"} err="failed to get container status \"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b\": rpc error: code = NotFound desc = could not find container \"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b\": container with ID starting with a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b not found: ID does not exist" Sep 15 07:06:10.395799 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.395762 2552 scope.go:117] "RemoveContainer" containerID="2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086" Sep 15 07:06:10.396068 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:10.396052 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086\": container with ID starting with 2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086 not found: ID does not exist" containerID="2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086" Sep 15 07:06:10.396133 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.396071 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086"} err="failed to get container status \"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086\": rpc error: code = NotFound desc = could not find container \"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086\": container with ID starting with 2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086 not found: ID does not exist" Sep 15 07:06:10.396133 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.396083 2552 scope.go:117] "RemoveContainer" containerID="974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c" Sep 15 07:06:10.396314 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:10.396298 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c\": container with ID starting with 974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c not found: ID does not exist" containerID="974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c" Sep 15 07:06:10.396351 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.396318 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c"} err="failed to get container status \"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c\": rpc error: code = NotFound desc = could not find container \"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c\": container with ID starting with 974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c not found: ID does not exist" Sep 15 07:06:10.396351 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.396331 2552 scope.go:117] "RemoveContainer" containerID="8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e" Sep 15 07:06:10.396590 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:10.396571 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e\": container with ID starting with 8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e not found: ID does not exist" containerID="8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e" Sep 15 07:06:10.396631 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.396598 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e"} err="failed to get container status \"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e\": rpc error: code = NotFound desc = could not find container \"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e\": container with ID starting with 8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e not found: ID does not exist" Sep 15 07:06:10.396631 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.396619 2552 scope.go:117] "RemoveContainer" containerID="b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e" Sep 15 07:06:10.396870 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:10.396852 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e\": container with ID starting with b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e not found: ID does not exist" containerID="b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e" Sep 15 07:06:10.396933 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.396877 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e"} err="failed to get container status \"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e\": rpc error: code = NotFound desc = could not find container \"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e\": container with ID starting with b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e not found: ID does not exist" Sep 15 07:06:10.396933 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.396899 2552 scope.go:117] "RemoveContainer" containerID="c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942" Sep 15 07:06:10.397184 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:10.397167 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942\": container with ID starting with c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942 not found: ID does not exist" containerID="c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942" Sep 15 07:06:10.397251 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.397188 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942"} err="failed to get container status \"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942\": rpc error: code = NotFound desc = could not find container \"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942\": container with ID starting with c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942 not found: ID does not exist" Sep 15 07:06:10.397251 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.397204 2552 scope.go:117] "RemoveContainer" containerID="a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b" Sep 15 07:06:10.397423 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.397405 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b"} err="failed to get container status \"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b\": rpc error: code = NotFound desc = could not find container \"a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b\": container with ID starting with a99d9773bf85aa911f17a22ce7104dc583c93c618d72f6633ef34ed86a248c6b not found: ID does not exist" Sep 15 07:06:10.397468 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.397424 2552 scope.go:117] "RemoveContainer" containerID="2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086" Sep 15 07:06:10.397639 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.397622 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086"} err="failed to get container status \"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086\": rpc error: code = NotFound desc = could not find container \"2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086\": container with ID starting with 2430656fce8b13de1564dd79366164b8056300cc362dd11fe2fdcc85941d0086 not found: ID does not exist" Sep 15 07:06:10.397695 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.397639 2552 scope.go:117] "RemoveContainer" containerID="974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c" Sep 15 07:06:10.397852 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.397836 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c"} err="failed to get container status \"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c\": rpc error: code = NotFound desc = could not find container \"974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c\": container with ID starting with 974312e36aaacd60db9e8f0c5f0b7fc9e4193497cdb312a25de6b08f730dfc7c not found: ID does not exist" Sep 15 07:06:10.397898 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.397852 2552 scope.go:117] "RemoveContainer" containerID="8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e" Sep 15 07:06:10.398070 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.398049 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e"} err="failed to get container status \"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e\": rpc error: code = NotFound desc = could not find container \"8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e\": container with ID starting with 8dde4d232c0f497d4b473911c70153dedda0a443af70962ff2f1f33969339c5e not found: ID does not exist" Sep 15 07:06:10.398139 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.398072 2552 scope.go:117] "RemoveContainer" containerID="b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e" Sep 15 07:06:10.398314 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.398291 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e"} err="failed to get container status \"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e\": rpc error: code = NotFound desc = could not find container \"b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e\": container with ID starting with b4f316e80e8f0c2ee8948a476ffe20f60d6ffd3b0006f477a6066d69607b3c3e not found: ID does not exist" Sep 15 07:06:10.398314 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.398309 2552 scope.go:117] "RemoveContainer" containerID="c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942" Sep 15 07:06:10.398522 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.398505 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942"} err="failed to get container status \"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942\": rpc error: code = NotFound desc = could not find container \"c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942\": container with ID starting with c05dbdba7b35f66b97572a3fd4d5b5f1cd5f4c4165859096b215f32ef3c9e942 not found: ID does not exist" Sep 15 07:06:10.398574 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.398524 2552 scope.go:117] "RemoveContainer" containerID="1721ae811e1df0c0083f9e3881c126862584cda7cbbbf21bb196e92286608995" Sep 15 07:06:10.405739 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.405722 2552 scope.go:117] "RemoveContainer" containerID="056910b482298820c063582c9e526f7d511f8c3e9f644d410c12b892dd069a37" Sep 15 07:06:10.413155 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.413138 2552 scope.go:117] "RemoveContainer" containerID="06f419db33738ab946a3f9cc24f9126b747160a133e4c600d60ea6ee76d7e5a1" Sep 15 07:06:10.420299 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.420278 2552 scope.go:117] "RemoveContainer" containerID="fb770377e99c315708d284bd1764503434b4750332e64cb4e53b40828d9cc45f" Sep 15 07:06:10.427371 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.427348 2552 scope.go:117] "RemoveContainer" containerID="59449698d5a6384c07e8490b603cc3d138094c1698ed4a25e90dd55e90ba0f20" Sep 15 07:06:10.861802 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.861770 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" path="/var/lib/kubelet/pods/a36c53a7-fafb-45e4-b803-7b14d983d665/volumes" Sep 15 07:06:10.862322 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:10.862307 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" path="/var/lib/kubelet/pods/be565ff7-465e-438c-bcb0-c13c7dbef853/volumes" Sep 15 07:06:26.817906 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:26.817865 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm"] Sep 15 07:06:26.818361 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:26.818075 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" podUID="2236bb72-97ed-4f31-b54e-4798d17e2d4c" containerName="manager" containerID="cri-o://5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234" gracePeriod=10 Sep 15 07:06:27.058152 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.058124 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:06:27.177512 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.177478 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/2236bb72-97ed-4f31-b54e-4798d17e2d4c-kube-api-access-ng6k8\") pod \"2236bb72-97ed-4f31-b54e-4798d17e2d4c\" (UID: \"2236bb72-97ed-4f31-b54e-4798d17e2d4c\") " Sep 15 07:06:27.179651 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.179622 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2236bb72-97ed-4f31-b54e-4798d17e2d4c-kube-api-access-ng6k8" (OuterVolumeSpecName: "kube-api-access-ng6k8") pod "2236bb72-97ed-4f31-b54e-4798d17e2d4c" (UID: "2236bb72-97ed-4f31-b54e-4798d17e2d4c"). InnerVolumeSpecName "kube-api-access-ng6k8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:06:27.278670 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.278629 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/2236bb72-97ed-4f31-b54e-4798d17e2d4c-kube-api-access-ng6k8\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:06:27.409074 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.409031 2552 generic.go:356] "Generic (PLEG): container finished" podID="2236bb72-97ed-4f31-b54e-4798d17e2d4c" containerID="5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234" exitCode=0 Sep 15 07:06:27.409074 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.409075 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" Sep 15 07:06:27.409277 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.409086 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" event={"ID":"2236bb72-97ed-4f31-b54e-4798d17e2d4c","Type":"ContainerDied","Data":"5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234"} Sep 15 07:06:27.409277 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.409135 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm" event={"ID":"2236bb72-97ed-4f31-b54e-4798d17e2d4c","Type":"ContainerDied","Data":"d271e2dbded67ac71034467844ec4638558975a7e7ab3fd116ea41c7e7cf4a13"} Sep 15 07:06:27.409277 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.409159 2552 scope.go:117] "RemoveContainer" containerID="5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234" Sep 15 07:06:27.417295 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.417271 2552 scope.go:117] "RemoveContainer" containerID="5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234" Sep 15 07:06:27.417592 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:27.417570 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234\": container with ID starting with 5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234 not found: ID does not exist" containerID="5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234" Sep 15 07:06:27.417658 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.417602 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234"} err="failed to get container status \"5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234\": rpc error: code = NotFound desc = could not find container \"5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234\": container with ID starting with 5c199b369e1a213cf85ff1d10858d2895b08520a4ac172f8d86cdf197d640234 not found: ID does not exist" Sep 15 07:06:27.431468 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.429732 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm"] Sep 15 07:06:27.433997 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:27.433296 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hx4mm"] Sep 15 07:06:28.860189 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:28.860154 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2236bb72-97ed-4f31-b54e-4798d17e2d4c" path="/var/lib/kubelet/pods/2236bb72-97ed-4f31-b54e-4798d17e2d4c/volumes" Sep 15 07:06:31.389446 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389367 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9"] Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389639 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389651 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389661 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2236bb72-97ed-4f31-b54e-4798d17e2d4c" containerName="manager" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389667 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2236bb72-97ed-4f31-b54e-4798d17e2d4c" containerName="manager" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389674 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="registry" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389679 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="registry" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389685 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389691 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389698 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389704 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389717 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="feast-apply" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389724 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="feast-apply" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389732 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389736 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389744 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="offline" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389748 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="offline" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389774 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="feast-init" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389780 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="feast-init" Sep 15 07:06:31.389784 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389788 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="feast-init" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389797 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="feast-init" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389816 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="feast-apply" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389821 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="feast-apply" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389827 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="offline" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389833 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="offline" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389882 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="offline" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389890 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="ui" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389896 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="ui" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389902 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="offline" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389909 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a36c53a7-fafb-45e4-b803-7b14d983d665" containerName="online" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389914 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2236bb72-97ed-4f31-b54e-4798d17e2d4c" containerName="manager" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389920 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="registry" Sep 15 07:06:31.390349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.389924 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="be565ff7-465e-438c-bcb0-c13c7dbef853" containerName="online" Sep 15 07:06:31.394307 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.394291 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" Sep 15 07:06:31.397588 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.397565 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-mdd4g\"" Sep 15 07:06:31.398760 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.398742 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 15 07:06:31.398861 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.398760 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 15 07:06:31.403320 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.403292 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9"] Sep 15 07:06:31.510953 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.510908 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrw9n\" (UniqueName: \"kubernetes.io/projected/b679f0d6-b4ca-44ba-ab42-9ed177f9056c-kube-api-access-zrw9n\") pod \"feast-operator-controller-manager-689b877b4b-hs5d9\" (UID: \"b679f0d6-b4ca-44ba-ab42-9ed177f9056c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" Sep 15 07:06:31.611786 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.611742 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrw9n\" (UniqueName: \"kubernetes.io/projected/b679f0d6-b4ca-44ba-ab42-9ed177f9056c-kube-api-access-zrw9n\") pod \"feast-operator-controller-manager-689b877b4b-hs5d9\" (UID: \"b679f0d6-b4ca-44ba-ab42-9ed177f9056c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" Sep 15 07:06:31.621086 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.621056 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrw9n\" (UniqueName: \"kubernetes.io/projected/b679f0d6-b4ca-44ba-ab42-9ed177f9056c-kube-api-access-zrw9n\") pod \"feast-operator-controller-manager-689b877b4b-hs5d9\" (UID: \"b679f0d6-b4ca-44ba-ab42-9ed177f9056c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" Sep 15 07:06:31.705320 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.705284 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" Sep 15 07:06:31.974815 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:31.974734 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9"] Sep 15 07:06:31.978468 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:06:31.978442 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb679f0d6_b4ca_44ba_ab42_9ed177f9056c.slice/crio-63f91206101da33e93d61e1b01c6be560e923fbe38d9229fd96b3b5d86755008 WatchSource:0}: Error finding container 63f91206101da33e93d61e1b01c6be560e923fbe38d9229fd96b3b5d86755008: Status 404 returned error can't find the container with id 63f91206101da33e93d61e1b01c6be560e923fbe38d9229fd96b3b5d86755008 Sep 15 07:06:32.428366 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:32.428331 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" event={"ID":"b679f0d6-b4ca-44ba-ab42-9ed177f9056c","Type":"ContainerStarted","Data":"a94edd7ce7ff3fb87a641f592be9ae9fd3d06d2d9f1805f0409791aacdd40642"} Sep 15 07:06:32.428788 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:32.428394 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" event={"ID":"b679f0d6-b4ca-44ba-ab42-9ed177f9056c","Type":"ContainerStarted","Data":"63f91206101da33e93d61e1b01c6be560e923fbe38d9229fd96b3b5d86755008"} Sep 15 07:06:32.428788 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:32.428454 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" Sep 15 07:06:32.444319 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:32.444256 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" podStartSLOduration=1.444233256 podStartE2EDuration="1.444233256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:06:32.444122378 +0000 UTC m=+1795.998779802" watchObservedRunningTime="2026-09-15 07:06:32.444233256 +0000 UTC m=+1795.998890668" Sep 15 07:06:43.434461 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:43.434431 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-hs5d9" Sep 15 07:06:49.113654 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.113618 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws"] Sep 15 07:06:49.117043 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.117023 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.122534 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.122506 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 15 07:06:49.122763 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.122747 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 15 07:06:49.122872 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.122850 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 15 07:06:49.123879 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.123859 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 15 07:06:49.123997 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.123917 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-jpvqw\"" Sep 15 07:06:49.123997 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.123945 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 15 07:06:49.123997 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.123955 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 15 07:06:49.140511 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.140481 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws"] Sep 15 07:06:49.252022 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.251955 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.252202 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.252061 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.252202 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.252080 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.252202 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.252118 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.252202 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.252180 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.252342 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.252214 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8fh6\" (UniqueName: \"kubernetes.io/projected/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-kube-api-access-l8fh6\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.352620 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.352587 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.352741 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.352623 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.352741 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.352643 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8fh6\" (UniqueName: \"kubernetes.io/projected/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-kube-api-access-l8fh6\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.352741 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.352664 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.352899 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:49.352752 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 15 07:06:49.352899 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.352779 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.352899 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.352814 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.352899 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:06:49.352842 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls podName:c1ada0a4-c5e0-4efc-a068-87bb6c0423da nodeName:}" failed. No retries permitted until 2026-09-15 07:06:49.852819621 +0000 UTC m=+1813.407477022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-b9jws" (UID: "c1ada0a4-c5e0-4efc-a068-87bb6c0423da") : secret "feast-simple-feast-setup-offline-tls" not found Sep 15 07:06:49.353119 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.353041 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.355204 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.355176 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.355327 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.355265 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.355327 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.355307 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.362773 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.362744 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8fh6\" (UniqueName: \"kubernetes.io/projected/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-kube-api-access-l8fh6\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.856334 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.856295 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:49.858672 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:49.858645 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-b9jws\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:50.026499 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:50.026462 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:06:50.304828 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:50.304793 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws"] Sep 15 07:06:50.305512 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:06:50.305488 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1ada0a4_c5e0_4efc_a068_87bb6c0423da.slice/crio-4b1c731da9d74c78fe9687dde2cc6d657a644b2f86548d6be9ce1be8ed286021 WatchSource:0}: Error finding container 4b1c731da9d74c78fe9687dde2cc6d657a644b2f86548d6be9ce1be8ed286021: Status 404 returned error can't find the container with id 4b1c731da9d74c78fe9687dde2cc6d657a644b2f86548d6be9ce1be8ed286021 Sep 15 07:06:50.495451 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:50.495406 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerStarted","Data":"73df259779a59a87cae26e2337011c260ae1dcd5e28f1eea94da5a1f38e7e6fa"} Sep 15 07:06:50.495624 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:50.495473 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerStarted","Data":"4b1c731da9d74c78fe9687dde2cc6d657a644b2f86548d6be9ce1be8ed286021"} Sep 15 07:06:55.517474 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:55.517440 2552 generic.go:356] "Generic (PLEG): container finished" podID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerID="73df259779a59a87cae26e2337011c260ae1dcd5e28f1eea94da5a1f38e7e6fa" exitCode=0 Sep 15 07:06:55.517854 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:55.517508 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerDied","Data":"73df259779a59a87cae26e2337011c260ae1dcd5e28f1eea94da5a1f38e7e6fa"} Sep 15 07:06:56.522922 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:06:56.522887 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerStarted","Data":"ff1b6484248e1c419fd710ca06919ea3ab1f304cea75f82cc1825e2652b3b39a"} Sep 15 07:07:02.546372 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:02.546339 2552 generic.go:356] "Generic (PLEG): container finished" podID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerID="ff1b6484248e1c419fd710ca06919ea3ab1f304cea75f82cc1825e2652b3b39a" exitCode=0 Sep 15 07:07:02.546760 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:02.546415 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerDied","Data":"ff1b6484248e1c419fd710ca06919ea3ab1f304cea75f82cc1825e2652b3b39a"} Sep 15 07:07:03.556431 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:03.556393 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerStarted","Data":"b8291f80aa330a8e59b4406684838b26b695e3d01d83507a35ee7a25c5df9f59"} Sep 15 07:07:03.556924 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:03.556456 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerStarted","Data":"422af6f623e0d3aef98af0a8604f681cbe28886e9d5d83e9e8b233401311520e"} Sep 15 07:07:03.556924 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:03.556475 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerStarted","Data":"02e1da714e813f00a777bd99eff1b4d03d7985e3c13abd15f0cbae9669c2833c"} Sep 15 07:07:03.556924 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:03.556489 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerStarted","Data":"c0495cc5b439f187795e93b29dbaf8c8fd65568638c7d0320e5a414683c6286c"} Sep 15 07:07:03.583022 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:03.582929 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podStartSLOduration=14.582904567 podStartE2EDuration="14.582904567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:07:03.580760611 +0000 UTC m=+1827.135418037" watchObservedRunningTime="2026-09-15 07:07:03.582904567 +0000 UTC m=+1827.137561991" Sep 15 07:07:05.027418 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:05.027378 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:05.027909 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:05.027445 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:05.027909 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:05.027466 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:05.027909 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:05.027485 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:05.029459 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:05.029419 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 15 07:07:05.029717 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:05.029441 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused" Sep 15 07:07:05.029857 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:05.029457 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 15 07:07:05.029857 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:05.029467 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 15 07:07:08.026983 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:08.026917 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused" Sep 15 07:07:08.026983 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:08.026939 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.30:6571: connect: connection refused" Sep 15 07:07:08.027547 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:08.027227 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 15 07:07:08.027660 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:08.027621 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 15 07:07:10.026884 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:10.026836 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:10.027497 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:10.027462 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:10.027497 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:10.027499 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:10.027682 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:10.027518 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:11.026750 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:11.026700 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 15 07:07:11.027820 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:11.027794 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:11.028225 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:11.028063 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:11.033271 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:11.033246 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:11.596537 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:11.596498 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:11.596701 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:11.596658 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:11.599206 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:11.599179 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:14.027685 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:14.027651 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:07:14.608201 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:07:14.608170 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:08:16.415827 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.415741 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws"] Sep 15 07:08:16.418873 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.416053 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="registry" containerID="cri-o://c0495cc5b439f187795e93b29dbaf8c8fd65568638c7d0320e5a414683c6286c" gracePeriod=30 Sep 15 07:08:16.418873 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.416090 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" containerID="cri-o://b8291f80aa330a8e59b4406684838b26b695e3d01d83507a35ee7a25c5df9f59" gracePeriod=30 Sep 15 07:08:16.418873 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.416116 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="offline" containerID="cri-o://422af6f623e0d3aef98af0a8604f681cbe28886e9d5d83e9e8b233401311520e" gracePeriod=30 Sep 15 07:08:16.418873 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.416120 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" containerID="cri-o://02e1da714e813f00a777bd99eff1b4d03d7985e3c13abd15f0cbae9669c2833c" gracePeriod=30 Sep 15 07:08:16.865873 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.865841 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq"] Sep 15 07:08:16.869364 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.869339 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:16.872319 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.872295 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-q7sgj\"" Sep 15 07:08:16.895196 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:16.895166 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq"] Sep 15 07:08:17.052302 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.052271 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.052489 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.052328 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.052489 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.052349 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.052489 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.052390 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5647c4e8-07d4-4399-8bad-13b85a7c2563-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.052489 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.052461 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.052645 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.052491 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmh4f\" (UniqueName: \"kubernetes.io/projected/5647c4e8-07d4-4399-8bad-13b85a7c2563-kube-api-access-lmh4f\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.153088 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.152995 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.153088 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.153038 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmh4f\" (UniqueName: \"kubernetes.io/projected/5647c4e8-07d4-4399-8bad-13b85a7c2563-kube-api-access-lmh4f\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.153088 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.153075 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.153363 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.153147 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.153363 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.153190 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.153363 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.153215 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5647c4e8-07d4-4399-8bad-13b85a7c2563-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.153588 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.153563 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5647c4e8-07d4-4399-8bad-13b85a7c2563-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.155768 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.155739 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.155886 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.155822 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.155954 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.155919 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.156037 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.155949 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.161792 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.161762 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmh4f\" (UniqueName: \"kubernetes.io/projected/5647c4e8-07d4-4399-8bad-13b85a7c2563-kube-api-access-lmh4f\") pod \"feast-simple-feast-setup-57576b8fcf-n28cq\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.178634 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.178594 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:17.466284 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.466260 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq"] Sep 15 07:08:17.468422 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:08:17.468395 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5647c4e8_07d4_4399_8bad_13b85a7c2563.slice/crio-8b449a2554921e5c8548009f7066cb635e4e1b8399da87c8b600108c6588bca3 WatchSource:0}: Error finding container 8b449a2554921e5c8548009f7066cb635e4e1b8399da87c8b600108c6588bca3: Status 404 returned error can't find the container with id 8b449a2554921e5c8548009f7066cb635e4e1b8399da87c8b600108c6588bca3 Sep 15 07:08:17.845847 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.845755 2552 generic.go:356] "Generic (PLEG): container finished" podID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerID="b8291f80aa330a8e59b4406684838b26b695e3d01d83507a35ee7a25c5df9f59" exitCode=0 Sep 15 07:08:17.845847 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.845782 2552 generic.go:356] "Generic (PLEG): container finished" podID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerID="02e1da714e813f00a777bd99eff1b4d03d7985e3c13abd15f0cbae9669c2833c" exitCode=0 Sep 15 07:08:17.845847 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.845834 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerDied","Data":"b8291f80aa330a8e59b4406684838b26b695e3d01d83507a35ee7a25c5df9f59"} Sep 15 07:08:17.846110 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.845878 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerDied","Data":"02e1da714e813f00a777bd99eff1b4d03d7985e3c13abd15f0cbae9669c2833c"} Sep 15 07:08:17.847158 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.847132 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerStarted","Data":"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199"} Sep 15 07:08:17.847252 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:17.847172 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerStarted","Data":"8b449a2554921e5c8548009f7066cb635e4e1b8399da87c8b600108c6588bca3"} Sep 15 07:08:21.596677 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:21.596630 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 15 07:08:22.868252 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:22.868219 2552 generic.go:356] "Generic (PLEG): container finished" podID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerID="05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199" exitCode=0 Sep 15 07:08:22.868626 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:22.868296 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerDied","Data":"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199"} Sep 15 07:08:23.874191 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:23.874160 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerStarted","Data":"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e"} Sep 15 07:08:24.608018 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:24.607948 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 15 07:08:28.897376 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:28.897342 2552 generic.go:356] "Generic (PLEG): container finished" podID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerID="69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e" exitCode=0 Sep 15 07:08:28.897742 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:28.897404 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerDied","Data":"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e"} Sep 15 07:08:29.907378 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:29.907330 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerStarted","Data":"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754"} Sep 15 07:08:29.907881 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:29.907391 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerStarted","Data":"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097"} Sep 15 07:08:29.907881 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:29.907419 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerStarted","Data":"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3"} Sep 15 07:08:29.907881 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:29.907438 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerStarted","Data":"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3"} Sep 15 07:08:29.936218 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:29.936144 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podStartSLOduration=13.936117908 podStartE2EDuration="13.936117908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:08:29.932746923 +0000 UTC m=+1913.487404338" watchObservedRunningTime="2026-09-15 07:08:29.936117908 +0000 UTC m=+1913.490775334" Sep 15 07:08:31.597064 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:31.597003 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 15 07:08:32.179856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:32.179779 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:32.179856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:32.179862 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:32.180153 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:32.179884 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:32.180153 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:32.179901 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:32.182025 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:32.181987 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.31:8816: connect: connection refused" Sep 15 07:08:32.182163 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:32.182020 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 15 07:08:32.182163 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:32.182026 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 15 07:08:32.182163 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:32.182085 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 15 07:08:34.608709 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:34.608659 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 15 07:08:35.179805 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:35.179753 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 15 07:08:35.180021 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:35.179820 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.31:8816: connect: connection refused" Sep 15 07:08:35.180021 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:35.179758 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 15 07:08:35.180271 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:35.180235 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused" Sep 15 07:08:37.178830 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:37.178796 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:37.179247 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:37.178874 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:37.179247 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:37.178891 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:37.179247 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:37.178987 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:38.180249 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:38.180216 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:38.180610 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:38.180347 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:38.180610 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:38.180581 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:38.185349 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:38.185327 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:38.953708 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:38.953677 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:38.953870 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:38.953738 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:38.953870 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:38.953791 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:38.956225 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:38.956201 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:08:41.596713 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:41.596663 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 15 07:08:41.597153 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:41.596817 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:08:44.607844 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:44.607799 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 15 07:08:44.608318 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:44.607991 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:08:46.992649 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:46.992522 2552 generic.go:356] "Generic (PLEG): container finished" podID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerID="422af6f623e0d3aef98af0a8604f681cbe28886e9d5d83e9e8b233401311520e" exitCode=137 Sep 15 07:08:46.992649 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:46.992549 2552 generic.go:356] "Generic (PLEG): container finished" podID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerID="c0495cc5b439f187795e93b29dbaf8c8fd65568638c7d0320e5a414683c6286c" exitCode=137 Sep 15 07:08:46.992649 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:46.992573 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerDied","Data":"422af6f623e0d3aef98af0a8604f681cbe28886e9d5d83e9e8b233401311520e"} Sep 15 07:08:46.992649 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:46.992631 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerDied","Data":"c0495cc5b439f187795e93b29dbaf8c8fd65568638c7d0320e5a414683c6286c"} Sep 15 07:08:47.055996 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.055956 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:08:47.103164 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.103136 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-ui-tls\") pod \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " Sep 15 07:08:47.103348 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.103175 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-feast-data\") pod \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " Sep 15 07:08:47.103348 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.103205 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls\") pod \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " Sep 15 07:08:47.103348 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.103250 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-registry-tls\") pod \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " Sep 15 07:08:47.103348 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.103277 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8fh6\" (UniqueName: \"kubernetes.io/projected/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-kube-api-access-l8fh6\") pod \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " Sep 15 07:08:47.103572 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.103465 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-online-tls\") pod \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\" (UID: \"c1ada0a4-c5e0-4efc-a068-87bb6c0423da\") " Sep 15 07:08:47.103708 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.103683 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-feast-data" (OuterVolumeSpecName: "feast-data") pod "c1ada0a4-c5e0-4efc-a068-87bb6c0423da" (UID: "c1ada0a4-c5e0-4efc-a068-87bb6c0423da"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:08:47.105498 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.105473 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c1ada0a4-c5e0-4efc-a068-87bb6c0423da" (UID: "c1ada0a4-c5e0-4efc-a068-87bb6c0423da"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:08:47.105782 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.105763 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-online-tls" (OuterVolumeSpecName: "online-tls") pod "c1ada0a4-c5e0-4efc-a068-87bb6c0423da" (UID: "c1ada0a4-c5e0-4efc-a068-87bb6c0423da"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:08:47.105853 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.105778 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c1ada0a4-c5e0-4efc-a068-87bb6c0423da" (UID: "c1ada0a4-c5e0-4efc-a068-87bb6c0423da"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:08:47.105913 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.105850 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-kube-api-access-l8fh6" (OuterVolumeSpecName: "kube-api-access-l8fh6") pod "c1ada0a4-c5e0-4efc-a068-87bb6c0423da" (UID: "c1ada0a4-c5e0-4efc-a068-87bb6c0423da"). InnerVolumeSpecName "kube-api-access-l8fh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:08:47.105913 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.105856 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c1ada0a4-c5e0-4efc-a068-87bb6c0423da" (UID: "c1ada0a4-c5e0-4efc-a068-87bb6c0423da"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:08:47.204632 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.204547 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-ui-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:08:47.204632 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.204573 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-feast-data\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:08:47.204632 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.204582 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-offline-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:08:47.204632 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.204590 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-registry-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:08:47.204632 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.204599 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8fh6\" (UniqueName: \"kubernetes.io/projected/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-kube-api-access-l8fh6\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:08:47.204632 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.204610 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c1ada0a4-c5e0-4efc-a068-87bb6c0423da-online-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:08:47.998983 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.998944 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" event={"ID":"c1ada0a4-c5e0-4efc-a068-87bb6c0423da","Type":"ContainerDied","Data":"4b1c731da9d74c78fe9687dde2cc6d657a644b2f86548d6be9ce1be8ed286021"} Sep 15 07:08:47.999389 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.999015 2552 scope.go:117] "RemoveContainer" containerID="b8291f80aa330a8e59b4406684838b26b695e3d01d83507a35ee7a25c5df9f59" Sep 15 07:08:47.999389 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:47.998955 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws" Sep 15 07:08:48.007607 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:48.007587 2552 scope.go:117] "RemoveContainer" containerID="422af6f623e0d3aef98af0a8604f681cbe28886e9d5d83e9e8b233401311520e" Sep 15 07:08:48.014740 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:48.014719 2552 scope.go:117] "RemoveContainer" containerID="02e1da714e813f00a777bd99eff1b4d03d7985e3c13abd15f0cbae9669c2833c" Sep 15 07:08:48.020625 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:48.020602 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws"] Sep 15 07:08:48.022936 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:48.022804 2552 scope.go:117] "RemoveContainer" containerID="c0495cc5b439f187795e93b29dbaf8c8fd65568638c7d0320e5a414683c6286c" Sep 15 07:08:48.024407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:48.024387 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-b9jws"] Sep 15 07:08:48.030330 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:48.030312 2552 scope.go:117] "RemoveContainer" containerID="ff1b6484248e1c419fd710ca06919ea3ab1f304cea75f82cc1825e2652b3b39a" Sep 15 07:08:48.037222 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:48.037206 2552 scope.go:117] "RemoveContainer" containerID="73df259779a59a87cae26e2337011c260ae1dcd5e28f1eea94da5a1f38e7e6fa" Sep 15 07:08:48.860702 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:08:48.860663 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" path="/var/lib/kubelet/pods/c1ada0a4-c5e0-4efc-a068-87bb6c0423da/volumes" Sep 15 07:09:41.444240 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444209 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89"] Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444462 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="registry" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444472 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="registry" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444481 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="feast-init" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444486 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="feast-init" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444492 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="feast-apply" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444498 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="feast-apply" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444505 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444510 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444519 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="offline" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444524 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="offline" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444531 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444535 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444583 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="registry" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444592 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="online" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444598 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="ui" Sep 15 07:09:41.444722 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.444605 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1ada0a4-c5e0-4efc-a068-87bb6c0423da" containerName="offline" Sep 15 07:09:41.447056 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.447040 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.450164 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.449975 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 15 07:09:41.450164 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.450034 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 15 07:09:41.450164 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.450062 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-4m567\"" Sep 15 07:09:41.450164 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.450160 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 15 07:09:41.451212 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.451183 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 15 07:09:41.451312 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.451217 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 15 07:09:41.451312 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.451242 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 15 07:09:41.461589 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.461567 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89"] Sep 15 07:09:41.543045 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.543011 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-652fq\" (UniqueName: \"kubernetes.io/projected/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-kube-api-access-652fq\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.543181 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.543060 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.543181 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.543119 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.543181 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.543140 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.543181 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.543167 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.543325 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.543191 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.643540 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.643508 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.643683 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.643551 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.643683 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.643568 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.643683 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.643640 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.643790 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.643701 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.643790 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.643758 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-652fq\" (UniqueName: \"kubernetes.io/projected/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-kube-api-access-652fq\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.644103 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.644075 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.644371 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.644352 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.646133 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.646107 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.646248 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.646207 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.646248 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.646211 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.652417 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.652393 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-652fq\" (UniqueName: \"kubernetes.io/projected/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-kube-api-access-652fq\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-rsd89\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:41.756175 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:41.756097 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:09:42.027196 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:42.027123 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89"] Sep 15 07:09:42.030294 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:09:42.030270 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7eb3c1b4_a947_4904_ae99_d25d12ea21b9.slice/crio-61c9c753a69d64669e89a6ba0d016930337d652d99e1467669cc2f28e3c7a76f WatchSource:0}: Error finding container 61c9c753a69d64669e89a6ba0d016930337d652d99e1467669cc2f28e3c7a76f: Status 404 returned error can't find the container with id 61c9c753a69d64669e89a6ba0d016930337d652d99e1467669cc2f28e3c7a76f Sep 15 07:09:42.202374 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:42.202338 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerStarted","Data":"4b89ae45cced743c5ac4eced4adcdc7d27748f18ba866f6167d85f401037b854"} Sep 15 07:09:42.202537 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:42.202381 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerStarted","Data":"61c9c753a69d64669e89a6ba0d016930337d652d99e1467669cc2f28e3c7a76f"} Sep 15 07:09:47.220897 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:47.220861 2552 generic.go:356] "Generic (PLEG): container finished" podID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerID="4b89ae45cced743c5ac4eced4adcdc7d27748f18ba866f6167d85f401037b854" exitCode=0 Sep 15 07:09:47.221304 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:47.220933 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerDied","Data":"4b89ae45cced743c5ac4eced4adcdc7d27748f18ba866f6167d85f401037b854"} Sep 15 07:09:48.226402 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:09:48.226366 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerStarted","Data":"d42fd975f7c43c6dd5111b71f9e8db7c6a53cddff3cfd13d5f561936a92a838b"} Sep 15 07:10:10.313669 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:10.313636 2552 generic.go:356] "Generic (PLEG): container finished" podID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerID="d42fd975f7c43c6dd5111b71f9e8db7c6a53cddff3cfd13d5f561936a92a838b" exitCode=0 Sep 15 07:10:10.314042 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:10.313708 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerDied","Data":"d42fd975f7c43c6dd5111b71f9e8db7c6a53cddff3cfd13d5f561936a92a838b"} Sep 15 07:10:11.329894 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.329847 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerStarted","Data":"512ca2044f7bf403f5a7a84015407dcd9ac9622d84e17e574f96db775c198c5e"} Sep 15 07:10:11.330276 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.329920 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerStarted","Data":"31e260cd549082e3a8b362a9b737fdba504f7e6f2b49529c2ed7d5157f9261b9"} Sep 15 07:10:11.330276 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.329940 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerStarted","Data":"b326727216b294b5097772cf6bb96d47092860361094bb5c4932d78c094ae389"} Sep 15 07:10:11.351861 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.351814 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podStartSLOduration=30.351800271 podStartE2EDuration="30.351800271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:10:11.350579129 +0000 UTC m=+2014.905236547" watchObservedRunningTime="2026-09-15 07:10:11.351800271 +0000 UTC m=+2014.906457693" Sep 15 07:10:11.757095 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.757035 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:11.757295 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.757130 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:11.757295 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.757201 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:11.757295 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.757219 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:11.757295 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.757236 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:11.757295 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.757253 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:11.758580 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.758538 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 15 07:10:11.758580 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.758565 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused" Sep 15 07:10:11.758759 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:11.758591 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 15 07:10:14.756569 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:14.756519 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused" Sep 15 07:10:14.757046 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:14.756581 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 15 07:10:14.757046 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:14.756533 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 15 07:10:17.756379 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:17.756332 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 15 07:10:17.756379 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:17.756375 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused" Sep 15 07:10:17.756910 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:17.756483 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 15 07:10:20.757150 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:20.757118 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:20.757626 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:20.757336 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:20.758144 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:20.758116 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:20.758252 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:20.758185 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:21.497643 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:21.497606 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:22.499465 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:22.499427 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:10:22.799177 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:22.799082 2552 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 15 07:10:23.503172 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:10:23.503130 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 15 07:11:23.587765 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.587677 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89"] Sep 15 07:11:23.588456 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.587928 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" containerID="cri-o://b326727216b294b5097772cf6bb96d47092860361094bb5c4932d78c094ae389" gracePeriod=30 Sep 15 07:11:23.588456 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.587960 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="offline" containerID="cri-o://31e260cd549082e3a8b362a9b737fdba504f7e6f2b49529c2ed7d5157f9261b9" gracePeriod=30 Sep 15 07:11:23.588456 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.587959 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" containerID="cri-o://512ca2044f7bf403f5a7a84015407dcd9ac9622d84e17e574f96db775c198c5e" gracePeriod=30 Sep 15 07:11:23.727984 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.727937 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq"] Sep 15 07:11:23.728228 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.728202 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="registry" containerID="cri-o://970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3" gracePeriod=30 Sep 15 07:11:23.728342 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.728314 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" containerID="cri-o://01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754" gracePeriod=30 Sep 15 07:11:23.728574 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.728329 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="offline" containerID="cri-o://9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097" gracePeriod=30 Sep 15 07:11:23.728681 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:23.728377 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" containerID="cri-o://dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3" gracePeriod=30 Sep 15 07:11:24.621003 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:24.620945 2552 generic.go:356] "Generic (PLEG): container finished" podID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerID="512ca2044f7bf403f5a7a84015407dcd9ac9622d84e17e574f96db775c198c5e" exitCode=0 Sep 15 07:11:24.621003 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:24.620993 2552 generic.go:356] "Generic (PLEG): container finished" podID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerID="b326727216b294b5097772cf6bb96d47092860361094bb5c4932d78c094ae389" exitCode=0 Sep 15 07:11:24.621452 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:24.621017 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerDied","Data":"512ca2044f7bf403f5a7a84015407dcd9ac9622d84e17e574f96db775c198c5e"} Sep 15 07:11:24.621452 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:24.621059 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerDied","Data":"b326727216b294b5097772cf6bb96d47092860361094bb5c4932d78c094ae389"} Sep 15 07:11:24.623922 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:24.623898 2552 generic.go:356] "Generic (PLEG): container finished" podID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerID="01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754" exitCode=0 Sep 15 07:11:24.623922 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:24.623919 2552 generic.go:356] "Generic (PLEG): container finished" podID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerID="dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3" exitCode=0 Sep 15 07:11:24.624079 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:24.623938 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerDied","Data":"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754"} Sep 15 07:11:24.624079 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:24.623978 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerDied","Data":"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3"} Sep 15 07:11:28.953281 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:28.953241 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 15 07:11:28.953701 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:28.953355 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 15 07:11:30.758269 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:30.758214 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 15 07:11:32.500345 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:32.500295 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 15 07:11:38.953935 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:38.953889 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 15 07:11:38.954430 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:38.953894 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 15 07:11:40.758032 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:40.757990 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 15 07:11:42.499991 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:42.499927 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 15 07:11:48.953680 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:48.953634 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.31:8443: connect: connection refused" Sep 15 07:11:48.954154 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:48.953652 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused" Sep 15 07:11:48.954154 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:48.953773 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:11:48.954154 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:48.953848 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:11:50.757956 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:50.757912 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.32:8443: connect: connection refused" Sep 15 07:11:50.758447 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:50.758061 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:11:52.500312 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:52.500269 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Sep 15 07:11:52.500727 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:52.500412 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:11:53.743726 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:53.743693 2552 generic.go:356] "Generic (PLEG): container finished" podID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerID="31e260cd549082e3a8b362a9b737fdba504f7e6f2b49529c2ed7d5157f9261b9" exitCode=137 Sep 15 07:11:53.744176 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:53.743763 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerDied","Data":"31e260cd549082e3a8b362a9b737fdba504f7e6f2b49529c2ed7d5157f9261b9"} Sep 15 07:11:54.234936 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.234914 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:11:54.270913 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.270880 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-652fq\" (UniqueName: \"kubernetes.io/projected/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-kube-api-access-652fq\") pod \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " Sep 15 07:11:54.270913 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.270916 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-feast-data\") pod \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " Sep 15 07:11:54.271252 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.270976 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-ui-tls\") pod \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " Sep 15 07:11:54.271252 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.271014 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-online-tls\") pod \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " Sep 15 07:11:54.271252 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.271034 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-registry-tls\") pod \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " Sep 15 07:11:54.271252 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.271060 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-offline-tls\") pod \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\" (UID: \"7eb3c1b4-a947-4904-ae99-d25d12ea21b9\") " Sep 15 07:11:54.272368 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.272319 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7eb3c1b4-a947-4904-ae99-d25d12ea21b9" (UID: "7eb3c1b4-a947-4904-ae99-d25d12ea21b9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 15 07:11:54.272630 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.272592 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-feast-data" (OuterVolumeSpecName: "feast-data") pod "7eb3c1b4-a947-4904-ae99-d25d12ea21b9" (UID: "7eb3c1b4-a947-4904-ae99-d25d12ea21b9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:11:54.273498 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.273475 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7eb3c1b4-a947-4904-ae99-d25d12ea21b9" (UID: "7eb3c1b4-a947-4904-ae99-d25d12ea21b9"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:11:54.273941 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.273915 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-online-tls" (OuterVolumeSpecName: "online-tls") pod "7eb3c1b4-a947-4904-ae99-d25d12ea21b9" (UID: "7eb3c1b4-a947-4904-ae99-d25d12ea21b9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:11:54.274070 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.273936 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-kube-api-access-652fq" (OuterVolumeSpecName: "kube-api-access-652fq") pod "7eb3c1b4-a947-4904-ae99-d25d12ea21b9" (UID: "7eb3c1b4-a947-4904-ae99-d25d12ea21b9"). InnerVolumeSpecName "kube-api-access-652fq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:11:54.274070 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.273936 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7eb3c1b4-a947-4904-ae99-d25d12ea21b9" (UID: "7eb3c1b4-a947-4904-ae99-d25d12ea21b9"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:11:54.371469 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.371439 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:11:54.371624 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.371604 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-652fq\" (UniqueName: \"kubernetes.io/projected/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-kube-api-access-652fq\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.371670 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.371627 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-feast-data\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.371670 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.371636 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-ui-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.371670 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.371644 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-online-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.371670 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.371651 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-registry-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.371670 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.371659 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7eb3c1b4-a947-4904-ae99-d25d12ea21b9-offline-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.472666 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.472624 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-online-tls\") pod \"5647c4e8-07d4-4399-8bad-13b85a7c2563\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " Sep 15 07:11:54.472845 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.472684 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmh4f\" (UniqueName: \"kubernetes.io/projected/5647c4e8-07d4-4399-8bad-13b85a7c2563-kube-api-access-lmh4f\") pod \"5647c4e8-07d4-4399-8bad-13b85a7c2563\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " Sep 15 07:11:54.472845 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.472748 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-registry-tls\") pod \"5647c4e8-07d4-4399-8bad-13b85a7c2563\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " Sep 15 07:11:54.472845 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.472776 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-ui-tls\") pod \"5647c4e8-07d4-4399-8bad-13b85a7c2563\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " Sep 15 07:11:54.473043 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.472913 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5647c4e8-07d4-4399-8bad-13b85a7c2563-feast-data\") pod \"5647c4e8-07d4-4399-8bad-13b85a7c2563\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " Sep 15 07:11:54.473043 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.472992 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-offline-tls\") pod \"5647c4e8-07d4-4399-8bad-13b85a7c2563\" (UID: \"5647c4e8-07d4-4399-8bad-13b85a7c2563\") " Sep 15 07:11:54.473527 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.473495 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5647c4e8-07d4-4399-8bad-13b85a7c2563-feast-data" (OuterVolumeSpecName: "feast-data") pod "5647c4e8-07d4-4399-8bad-13b85a7c2563" (UID: "5647c4e8-07d4-4399-8bad-13b85a7c2563"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:11:54.474938 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.474908 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5647c4e8-07d4-4399-8bad-13b85a7c2563" (UID: "5647c4e8-07d4-4399-8bad-13b85a7c2563"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:11:54.474938 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.474917 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5647c4e8-07d4-4399-8bad-13b85a7c2563-kube-api-access-lmh4f" (OuterVolumeSpecName: "kube-api-access-lmh4f") pod "5647c4e8-07d4-4399-8bad-13b85a7c2563" (UID: "5647c4e8-07d4-4399-8bad-13b85a7c2563"). InnerVolumeSpecName "kube-api-access-lmh4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:11:54.475103 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.474998 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5647c4e8-07d4-4399-8bad-13b85a7c2563" (UID: "5647c4e8-07d4-4399-8bad-13b85a7c2563"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:11:54.475194 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.475178 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "5647c4e8-07d4-4399-8bad-13b85a7c2563" (UID: "5647c4e8-07d4-4399-8bad-13b85a7c2563"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:11:54.475231 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.475221 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-online-tls" (OuterVolumeSpecName: "online-tls") pod "5647c4e8-07d4-4399-8bad-13b85a7c2563" (UID: "5647c4e8-07d4-4399-8bad-13b85a7c2563"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 15 07:11:54.573728 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.573695 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-registry-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.573728 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.573723 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-ui-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.573728 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.573733 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5647c4e8-07d4-4399-8bad-13b85a7c2563-feast-data\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.574056 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.573740 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-offline-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.574056 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.573748 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5647c4e8-07d4-4399-8bad-13b85a7c2563-online-tls\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.574056 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.573755 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmh4f\" (UniqueName: \"kubernetes.io/projected/5647c4e8-07d4-4399-8bad-13b85a7c2563-kube-api-access-lmh4f\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:11:54.749865 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.749834 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" Sep 15 07:11:54.750347 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.749835 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89" event={"ID":"7eb3c1b4-a947-4904-ae99-d25d12ea21b9","Type":"ContainerDied","Data":"61c9c753a69d64669e89a6ba0d016930337d652d99e1467669cc2f28e3c7a76f"} Sep 15 07:11:54.750347 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.749983 2552 scope.go:117] "RemoveContainer" containerID="512ca2044f7bf403f5a7a84015407dcd9ac9622d84e17e574f96db775c198c5e" Sep 15 07:11:54.752923 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.752896 2552 generic.go:356] "Generic (PLEG): container finished" podID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerID="9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097" exitCode=137 Sep 15 07:11:54.753045 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.752924 2552 generic.go:356] "Generic (PLEG): container finished" podID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerID="970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3" exitCode=137 Sep 15 07:11:54.753045 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.752995 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerDied","Data":"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097"} Sep 15 07:11:54.753045 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.753012 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" Sep 15 07:11:54.753045 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.753037 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerDied","Data":"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3"} Sep 15 07:11:54.753209 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.753061 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq" event={"ID":"5647c4e8-07d4-4399-8bad-13b85a7c2563","Type":"ContainerDied","Data":"8b449a2554921e5c8548009f7066cb635e4e1b8399da87c8b600108c6588bca3"} Sep 15 07:11:54.759669 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.759352 2552 scope.go:117] "RemoveContainer" containerID="31e260cd549082e3a8b362a9b737fdba504f7e6f2b49529c2ed7d5157f9261b9" Sep 15 07:11:54.767164 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.767147 2552 scope.go:117] "RemoveContainer" containerID="b326727216b294b5097772cf6bb96d47092860361094bb5c4932d78c094ae389" Sep 15 07:11:54.774199 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.774178 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89"] Sep 15 07:11:54.774632 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.774617 2552 scope.go:117] "RemoveContainer" containerID="d42fd975f7c43c6dd5111b71f9e8db7c6a53cddff3cfd13d5f561936a92a838b" Sep 15 07:11:54.777338 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.777319 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-rsd89"] Sep 15 07:11:54.781689 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.781673 2552 scope.go:117] "RemoveContainer" containerID="4b89ae45cced743c5ac4eced4adcdc7d27748f18ba866f6167d85f401037b854" Sep 15 07:11:54.788901 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.788883 2552 scope.go:117] "RemoveContainer" containerID="01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754" Sep 15 07:11:54.789669 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.789648 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq"] Sep 15 07:11:54.794988 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.794950 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-n28cq"] Sep 15 07:11:54.795937 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.795922 2552 scope.go:117] "RemoveContainer" containerID="9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097" Sep 15 07:11:54.802678 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.802662 2552 scope.go:117] "RemoveContainer" containerID="dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3" Sep 15 07:11:54.809471 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.809453 2552 scope.go:117] "RemoveContainer" containerID="970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3" Sep 15 07:11:54.816126 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.816107 2552 scope.go:117] "RemoveContainer" containerID="69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e" Sep 15 07:11:54.822567 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.822552 2552 scope.go:117] "RemoveContainer" containerID="05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199" Sep 15 07:11:54.829168 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.829151 2552 scope.go:117] "RemoveContainer" containerID="01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754" Sep 15 07:11:54.829431 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:11:54.829414 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754\": container with ID starting with 01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754 not found: ID does not exist" containerID="01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754" Sep 15 07:11:54.829486 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.829439 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754"} err="failed to get container status \"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754\": rpc error: code = NotFound desc = could not find container \"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754\": container with ID starting with 01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754 not found: ID does not exist" Sep 15 07:11:54.829486 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.829458 2552 scope.go:117] "RemoveContainer" containerID="9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097" Sep 15 07:11:54.829717 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:11:54.829700 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097\": container with ID starting with 9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097 not found: ID does not exist" containerID="9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097" Sep 15 07:11:54.829768 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.829720 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097"} err="failed to get container status \"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097\": rpc error: code = NotFound desc = could not find container \"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097\": container with ID starting with 9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097 not found: ID does not exist" Sep 15 07:11:54.829768 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.829733 2552 scope.go:117] "RemoveContainer" containerID="dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3" Sep 15 07:11:54.829945 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:11:54.829931 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3\": container with ID starting with dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3 not found: ID does not exist" containerID="dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3" Sep 15 07:11:54.830011 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.829948 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3"} err="failed to get container status \"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3\": rpc error: code = NotFound desc = could not find container \"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3\": container with ID starting with dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3 not found: ID does not exist" Sep 15 07:11:54.830011 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.829960 2552 scope.go:117] "RemoveContainer" containerID="970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3" Sep 15 07:11:54.830172 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:11:54.830157 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3\": container with ID starting with 970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3 not found: ID does not exist" containerID="970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3" Sep 15 07:11:54.830215 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.830176 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3"} err="failed to get container status \"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3\": rpc error: code = NotFound desc = could not find container \"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3\": container with ID starting with 970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3 not found: ID does not exist" Sep 15 07:11:54.830215 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.830189 2552 scope.go:117] "RemoveContainer" containerID="69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e" Sep 15 07:11:54.830427 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:11:54.830412 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e\": container with ID starting with 69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e not found: ID does not exist" containerID="69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e" Sep 15 07:11:54.830479 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.830428 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e"} err="failed to get container status \"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e\": rpc error: code = NotFound desc = could not find container \"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e\": container with ID starting with 69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e not found: ID does not exist" Sep 15 07:11:54.830479 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.830440 2552 scope.go:117] "RemoveContainer" containerID="05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199" Sep 15 07:11:54.830681 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:11:54.830666 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199\": container with ID starting with 05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199 not found: ID does not exist" containerID="05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199" Sep 15 07:11:54.830727 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.830685 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199"} err="failed to get container status \"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199\": rpc error: code = NotFound desc = could not find container \"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199\": container with ID starting with 05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199 not found: ID does not exist" Sep 15 07:11:54.830727 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.830699 2552 scope.go:117] "RemoveContainer" containerID="01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754" Sep 15 07:11:54.830885 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.830869 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754"} err="failed to get container status \"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754\": rpc error: code = NotFound desc = could not find container \"01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754\": container with ID starting with 01ab3f19b94f8139436e67b0d8617dbe922bcb670d39efcb89ce3601b8189754 not found: ID does not exist" Sep 15 07:11:54.830927 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.830884 2552 scope.go:117] "RemoveContainer" containerID="9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097" Sep 15 07:11:54.831104 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831085 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097"} err="failed to get container status \"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097\": rpc error: code = NotFound desc = could not find container \"9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097\": container with ID starting with 9d10ac8554b4d831e98c2d8a41bb100214e6a83059aae440db4c52b1b9850097 not found: ID does not exist" Sep 15 07:11:54.831162 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831107 2552 scope.go:117] "RemoveContainer" containerID="dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3" Sep 15 07:11:54.831313 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831293 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3"} err="failed to get container status \"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3\": rpc error: code = NotFound desc = could not find container \"dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3\": container with ID starting with dc9e5bee5207ef289f8dfec4750e33d769b1ae7da2cd95339fca19ab8072c8e3 not found: ID does not exist" Sep 15 07:11:54.831313 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831311 2552 scope.go:117] "RemoveContainer" containerID="970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3" Sep 15 07:11:54.831521 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831501 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3"} err="failed to get container status \"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3\": rpc error: code = NotFound desc = could not find container \"970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3\": container with ID starting with 970dc53d49ca76e3a6646789c46f7f490f6ae526e55887ada066cc8572c7c0f3 not found: ID does not exist" Sep 15 07:11:54.831569 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831523 2552 scope.go:117] "RemoveContainer" containerID="69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e" Sep 15 07:11:54.831734 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831716 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e"} err="failed to get container status \"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e\": rpc error: code = NotFound desc = could not find container \"69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e\": container with ID starting with 69b2517f9fdfe7c5209f7a8bd49f59b10fa978ae2ac97c1cf6881957dc76a72e not found: ID does not exist" Sep 15 07:11:54.831808 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831736 2552 scope.go:117] "RemoveContainer" containerID="05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199" Sep 15 07:11:54.831960 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.831941 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199"} err="failed to get container status \"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199\": rpc error: code = NotFound desc = could not find container \"05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199\": container with ID starting with 05721ac1b38bf45499e4f00294f5c47cb9a157c0858d4048ca24b22d4df65199 not found: ID does not exist" Sep 15 07:11:54.860410 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.860388 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" path="/var/lib/kubelet/pods/5647c4e8-07d4-4399-8bad-13b85a7c2563/volumes" Sep 15 07:11:54.861004 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:11:54.860988 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" path="/var/lib/kubelet/pods/7eb3c1b4-a947-4904-ae99-d25d12ea21b9/volumes" Sep 15 07:12:13.268407 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268370 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tgnpg/must-gather-7ghxf"] Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268635 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="offline" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268647 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="offline" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268656 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268662 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268672 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="offline" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268678 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="offline" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268687 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="feast-init" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268693 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="feast-init" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268701 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="registry" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268706 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="registry" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268714 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="feast-apply" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268719 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="feast-apply" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268726 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="feast-init" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268731 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="feast-init" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268738 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268743 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268748 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268753 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268761 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="feast-apply" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268766 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="feast-apply" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268772 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268777 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268814 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="offline" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268823 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="registry" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268828 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="offline" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268835 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="ui" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268839 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="online" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268845 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5647c4e8-07d4-4399-8bad-13b85a7c2563" containerName="online" Sep 15 07:12:13.268856 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.268851 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eb3c1b4-a947-4904-ae99-d25d12ea21b9" containerName="ui" Sep 15 07:12:13.271504 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.271487 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:13.274151 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.274132 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tgnpg\"/\"default-dockercfg-7kkvh\"" Sep 15 07:12:13.274151 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.274143 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tgnpg\"/\"openshift-service-ca.crt\"" Sep 15 07:12:13.275228 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.275215 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tgnpg\"/\"kube-root-ca.crt\"" Sep 15 07:12:13.280010 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.279988 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tgnpg/must-gather-7ghxf"] Sep 15 07:12:13.317305 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.317273 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/931b85e9-98b0-474c-8025-54cc6a31720a-must-gather-output\") pod \"must-gather-7ghxf\" (UID: \"931b85e9-98b0-474c-8025-54cc6a31720a\") " pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:13.317431 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.317332 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-646kp\" (UniqueName: \"kubernetes.io/projected/931b85e9-98b0-474c-8025-54cc6a31720a-kube-api-access-646kp\") pod \"must-gather-7ghxf\" (UID: \"931b85e9-98b0-474c-8025-54cc6a31720a\") " pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:13.418752 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.418721 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/931b85e9-98b0-474c-8025-54cc6a31720a-must-gather-output\") pod \"must-gather-7ghxf\" (UID: \"931b85e9-98b0-474c-8025-54cc6a31720a\") " pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:13.418899 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.418791 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-646kp\" (UniqueName: \"kubernetes.io/projected/931b85e9-98b0-474c-8025-54cc6a31720a-kube-api-access-646kp\") pod \"must-gather-7ghxf\" (UID: \"931b85e9-98b0-474c-8025-54cc6a31720a\") " pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:13.419072 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.419052 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/931b85e9-98b0-474c-8025-54cc6a31720a-must-gather-output\") pod \"must-gather-7ghxf\" (UID: \"931b85e9-98b0-474c-8025-54cc6a31720a\") " pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:13.426482 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.426459 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-646kp\" (UniqueName: \"kubernetes.io/projected/931b85e9-98b0-474c-8025-54cc6a31720a-kube-api-access-646kp\") pod \"must-gather-7ghxf\" (UID: \"931b85e9-98b0-474c-8025-54cc6a31720a\") " pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:13.581347 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.581266 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:13.847687 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.847662 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tgnpg/must-gather-7ghxf"] Sep 15 07:12:13.849784 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:12:13.849757 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod931b85e9_98b0_474c_8025_54cc6a31720a.slice/crio-bf8a5ba02ffa598180e8c7a85e19185306b02f23d49d62f7b13e82eafbd2914a WatchSource:0}: Error finding container bf8a5ba02ffa598180e8c7a85e19185306b02f23d49d62f7b13e82eafbd2914a: Status 404 returned error can't find the container with id bf8a5ba02ffa598180e8c7a85e19185306b02f23d49d62f7b13e82eafbd2914a Sep 15 07:12:13.851536 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:13.851519 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 15 07:12:14.838318 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:14.838273 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" event={"ID":"931b85e9-98b0-474c-8025-54cc6a31720a","Type":"ContainerStarted","Data":"bf8a5ba02ffa598180e8c7a85e19185306b02f23d49d62f7b13e82eafbd2914a"} Sep 15 07:12:20.864465 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:20.864425 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" event={"ID":"931b85e9-98b0-474c-8025-54cc6a31720a","Type":"ContainerStarted","Data":"ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc"} Sep 15 07:12:20.864465 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:20.864466 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" event={"ID":"931b85e9-98b0-474c-8025-54cc6a31720a","Type":"ContainerStarted","Data":"21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba"} Sep 15 07:12:20.879312 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:20.879259 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" podStartSLOduration=1.780691339 podStartE2EDuration="7.879240767s" totalImagesPullingTime="6.098549428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 07:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:12:20.879137823 +0000 UTC m=+2144.433795248" watchObservedRunningTime="2026-09-15 07:12:20.879240767 +0000 UTC m=+2144.433898196" Sep 15 07:12:29.898091 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:29.897961 2552 generic.go:356] "Generic (PLEG): container finished" podID="931b85e9-98b0-474c-8025-54cc6a31720a" containerID="21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba" exitCode=0 Sep 15 07:12:29.898091 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:29.898001 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" event={"ID":"931b85e9-98b0-474c-8025-54cc6a31720a","Type":"ContainerDied","Data":"21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba"} Sep 15 07:12:29.898510 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:29.898281 2552 scope.go:117] "RemoveContainer" containerID="21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba" Sep 15 07:12:30.543190 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:30.543156 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tgnpg_must-gather-7ghxf_931b85e9-98b0-474c-8025-54cc6a31720a/gather/0.log" Sep 15 07:12:33.840580 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:33.840548 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-96qh7_0401d78e-6529-4987-90cb-4369375d67be/global-pull-secret-syncer/0.log" Sep 15 07:12:33.947432 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:33.947397 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vtltd_8ae71f33-68fa-4077-8b65-38ff766cddc1/konnectivity-agent/0.log" Sep 15 07:12:34.015054 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:34.015025 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-232.ec2.internal_ed80a124298f3affca688f00871d8528/haproxy/0.log" Sep 15 07:12:35.853291 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:35.853255 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-tgnpg/must-gather-7ghxf"] Sep 15 07:12:35.853766 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:35.853436 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" containerName="copy" containerID="cri-o://ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc" gracePeriod=2 Sep 15 07:12:35.859037 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:35.859014 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-tgnpg/must-gather-7ghxf"] Sep 15 07:12:35.860830 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:12:35.860798 2552 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"931b85e9-98b0-474c-8025-54cc6a31720a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"openshift-must-gather-tgnpg\"/\"must-gather-7ghxf\": pods \"must-gather-7ghxf\" not found" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:36.073097 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.073075 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tgnpg_must-gather-7ghxf_931b85e9-98b0-474c-8025-54cc6a31720a/copy/0.log" Sep 15 07:12:36.074689 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.074674 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:36.077079 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:12:36.077053 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7ghxf\" is forbidden: User \"system:node:ip-10-0-133-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tgnpg\": no relationship found between node 'ip-10-0-133-232.ec2.internal' and this object" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:36.203002 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.202956 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/931b85e9-98b0-474c-8025-54cc6a31720a-must-gather-output\") pod \"931b85e9-98b0-474c-8025-54cc6a31720a\" (UID: \"931b85e9-98b0-474c-8025-54cc6a31720a\") " Sep 15 07:12:36.203177 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.203013 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-646kp\" (UniqueName: \"kubernetes.io/projected/931b85e9-98b0-474c-8025-54cc6a31720a-kube-api-access-646kp\") pod \"931b85e9-98b0-474c-8025-54cc6a31720a\" (UID: \"931b85e9-98b0-474c-8025-54cc6a31720a\") " Sep 15 07:12:36.203410 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.203377 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/931b85e9-98b0-474c-8025-54cc6a31720a-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "931b85e9-98b0-474c-8025-54cc6a31720a" (UID: "931b85e9-98b0-474c-8025-54cc6a31720a"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 15 07:12:36.205343 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.205314 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/931b85e9-98b0-474c-8025-54cc6a31720a-kube-api-access-646kp" (OuterVolumeSpecName: "kube-api-access-646kp") pod "931b85e9-98b0-474c-8025-54cc6a31720a" (UID: "931b85e9-98b0-474c-8025-54cc6a31720a"). InnerVolumeSpecName "kube-api-access-646kp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 15 07:12:36.303723 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.303684 2552 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/931b85e9-98b0-474c-8025-54cc6a31720a-must-gather-output\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:12:36.303723 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.303716 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-646kp\" (UniqueName: \"kubernetes.io/projected/931b85e9-98b0-474c-8025-54cc6a31720a-kube-api-access-646kp\") on node \"ip-10-0-133-232.ec2.internal\" DevicePath \"\"" Sep 15 07:12:36.859587 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:12:36.859540 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-7ghxf\" is forbidden: User \"system:node:ip-10-0-133-232.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tgnpg\": no relationship found between node 'ip-10-0-133-232.ec2.internal' and this object" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:36.861348 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.861318 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" path="/var/lib/kubelet/pods/931b85e9-98b0-474c-8025-54cc6a31720a/volumes" Sep 15 07:12:36.926581 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.926556 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tgnpg_must-gather-7ghxf_931b85e9-98b0-474c-8025-54cc6a31720a/copy/0.log" Sep 15 07:12:36.926898 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.926876 2552 generic.go:356] "Generic (PLEG): container finished" podID="931b85e9-98b0-474c-8025-54cc6a31720a" containerID="ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc" exitCode=143 Sep 15 07:12:36.927008 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.926923 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tgnpg/must-gather-7ghxf" Sep 15 07:12:36.927008 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.926957 2552 scope.go:117] "RemoveContainer" containerID="ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc" Sep 15 07:12:36.934157 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.934138 2552 scope.go:117] "RemoveContainer" containerID="21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba" Sep 15 07:12:36.945801 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.945782 2552 scope.go:117] "RemoveContainer" containerID="ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc" Sep 15 07:12:36.946126 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:12:36.946105 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc\": container with ID starting with ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc not found: ID does not exist" containerID="ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc" Sep 15 07:12:36.946224 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.946138 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc"} err="failed to get container status \"ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc\": rpc error: code = NotFound desc = could not find container \"ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc\": container with ID starting with ed2e1f2faffa01b3e213079fd72087bc7a871e4d1e92774979452a36fdc5aacc not found: ID does not exist" Sep 15 07:12:36.946224 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.946164 2552 scope.go:117] "RemoveContainer" containerID="21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba" Sep 15 07:12:36.946443 ip-10-0-133-232 kubenswrapper[2552]: E0915 07:12:36.946422 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba\": container with ID starting with 21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba not found: ID does not exist" containerID="21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba" Sep 15 07:12:36.946481 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:36.946451 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba"} err="failed to get container status \"21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba\": rpc error: code = NotFound desc = could not find container \"21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba\": container with ID starting with 21061ddb8da764d9fc7c66d4a3381f3473f20ed46774ae93a22adbe2949ed2ba not found: ID does not exist" Sep 15 07:12:37.759466 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:37.759437 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-2tg7m_82de95cc-b9c4-4546-b870-588edc3da907/cluster-monitoring-operator/0.log" Sep 15 07:12:37.794242 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:37.794218 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-tz6fd_a7ca278c-776f-45ce-abc1-9416bb615cd0/kube-state-metrics/0.log" Sep 15 07:12:37.820706 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:37.820680 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-tz6fd_a7ca278c-776f-45ce-abc1-9416bb615cd0/kube-rbac-proxy-main/0.log" Sep 15 07:12:37.848802 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:37.848775 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-tz6fd_a7ca278c-776f-45ce-abc1-9416bb615cd0/kube-rbac-proxy-self/0.log" Sep 15 07:12:37.912940 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:37.912912 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-rhbx5_75cc5b37-f8e9-481f-a034-16729e6ceb07/monitoring-plugin/0.log" Sep 15 07:12:37.945115 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:37.945091 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-622vl_af51d317-972e-49c1-8e27-a77632a4fa97/node-exporter/0.log" Sep 15 07:12:37.971479 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:37.971453 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-622vl_af51d317-972e-49c1-8e27-a77632a4fa97/kube-rbac-proxy/0.log" Sep 15 07:12:37.992660 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:37.992639 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-622vl_af51d317-972e-49c1-8e27-a77632a4fa97/init-textfile/0.log" Sep 15 07:12:38.454446 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:38.454416 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hx6k5_c489087f-e151-472a-a419-6a1791d8b5d4/prometheus-operator/0.log" Sep 15 07:12:38.475664 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:38.475636 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-hx6k5_c489087f-e151-472a-a419-6a1791d8b5d4/kube-rbac-proxy/0.log" Sep 15 07:12:40.889173 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:40.889142 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-gw6xp_bad6582f-6851-4bae-91f6-ad033be4414d/download-server/0.log" Sep 15 07:12:41.465988 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.465940 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s"] Sep 15 07:12:41.466329 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.466303 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" containerName="gather" Sep 15 07:12:41.466669 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.466478 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" containerName="gather" Sep 15 07:12:41.466669 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.466537 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" containerName="copy" Sep 15 07:12:41.466669 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.466551 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" containerName="copy" Sep 15 07:12:41.466669 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.466650 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" containerName="gather" Sep 15 07:12:41.466669 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.466661 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="931b85e9-98b0-474c-8025-54cc6a31720a" containerName="copy" Sep 15 07:12:41.470211 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.470191 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.472662 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.472644 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kgbcs\"/\"openshift-service-ca.crt\"" Sep 15 07:12:41.473864 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.473847 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kgbcs\"/\"default-dockercfg-89p9m\"" Sep 15 07:12:41.473947 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.473926 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kgbcs\"/\"kube-root-ca.crt\"" Sep 15 07:12:41.478729 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.478704 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s"] Sep 15 07:12:41.646296 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.646256 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnlww\" (UniqueName: \"kubernetes.io/projected/156cc80b-f17d-4253-bee2-46a544afa469-kube-api-access-rnlww\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.646451 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.646337 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-sys\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.646451 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.646364 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-lib-modules\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.646451 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.646383 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-podres\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.646588 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.646479 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-proc\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.746938 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.746846 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-proc\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.746938 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.746892 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnlww\" (UniqueName: \"kubernetes.io/projected/156cc80b-f17d-4253-bee2-46a544afa469-kube-api-access-rnlww\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.747146 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.746954 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-sys\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.747146 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.746997 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-proc\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.747146 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.747034 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-sys\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.747146 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.747089 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-lib-modules\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.747146 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.747132 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-podres\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.747314 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.747243 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-lib-modules\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.747314 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.747267 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/156cc80b-f17d-4253-bee2-46a544afa469-podres\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.755404 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.755371 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnlww\" (UniqueName: \"kubernetes.io/projected/156cc80b-f17d-4253-bee2-46a544afa469-kube-api-access-rnlww\") pod \"perf-node-gather-daemonset-jwb6s\" (UID: \"156cc80b-f17d-4253-bee2-46a544afa469\") " pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:41.779787 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:41.779768 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:42.048053 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.047959 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s"] Sep 15 07:12:42.051651 ip-10-0-133-232 kubenswrapper[2552]: W0915 07:12:42.051622 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod156cc80b_f17d_4253_bee2_46a544afa469.slice/crio-ff75dabafd3c21da628dac99886f564880ecc515c58253992a667620886744db WatchSource:0}: Error finding container ff75dabafd3c21da628dac99886f564880ecc515c58253992a667620886744db: Status 404 returned error can't find the container with id ff75dabafd3c21da628dac99886f564880ecc515c58253992a667620886744db Sep 15 07:12:42.171404 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.171370 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6nc7q_93f7caf1-70fc-43c2-b8f9-9710fbbf2cea/dns/0.log" Sep 15 07:12:42.195492 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.195468 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6nc7q_93f7caf1-70fc-43c2-b8f9-9710fbbf2cea/kube-rbac-proxy/0.log" Sep 15 07:12:42.278909 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.278883 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m48nb_27055683-09d8-4071-b297-5feb0eb4e3b8/dns-node-resolver/0.log" Sep 15 07:12:42.689239 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.689206 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-59bf595485-j25wn_d9c8c85a-4fa4-43f7-a842-e8c3370dfa2d/registry/0.log" Sep 15 07:12:42.732670 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.732641 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pjt59_a0e87755-014a-4616-b966-b66c76c86721/node-ca/0.log" Sep 15 07:12:42.949486 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.949398 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" event={"ID":"156cc80b-f17d-4253-bee2-46a544afa469","Type":"ContainerStarted","Data":"da63481015a96f006687fb05535ddbebd73f64a4c2ef070ed7dd5b1b120a530b"} Sep 15 07:12:42.949486 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.949453 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" event={"ID":"156cc80b-f17d-4253-bee2-46a544afa469","Type":"ContainerStarted","Data":"ff75dabafd3c21da628dac99886f564880ecc515c58253992a667620886744db"} Sep 15 07:12:42.949667 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.949540 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:42.965730 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:42.965664 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" podStartSLOduration=1.965646087 podStartE2EDuration="1.965646087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 07:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 07:12:42.965192507 +0000 UTC m=+2166.519849931" watchObservedRunningTime="2026-09-15 07:12:42.965646087 +0000 UTC m=+2166.520303493" Sep 15 07:12:43.698428 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:43.698398 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6dtk4_f8dda479-a879-433e-b4ca-b667b60c8c01/serve-healthcheck-canary/0.log" Sep 15 07:12:44.076962 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:44.076888 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84cfd_3d9de86b-d55c-49ef-83a8-677097aa9444/kube-rbac-proxy/0.log" Sep 15 07:12:44.097060 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:44.097030 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84cfd_3d9de86b-d55c-49ef-83a8-677097aa9444/exporter/0.log" Sep 15 07:12:44.119422 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:44.119392 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-84cfd_3d9de86b-d55c-49ef-83a8-677097aa9444/extractor/0.log" Sep 15 07:12:48.962287 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:48.962261 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kgbcs/perf-node-gather-daemonset-jwb6s" Sep 15 07:12:49.411930 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:49.411851 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hddd_178866a3-7693-4a4b-8a07-c489b6fed8a7/kube-multus-additional-cni-plugins/0.log" Sep 15 07:12:49.435687 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:49.435660 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hddd_178866a3-7693-4a4b-8a07-c489b6fed8a7/egress-router-binary-copy/0.log" Sep 15 07:12:49.459441 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:49.459419 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hddd_178866a3-7693-4a4b-8a07-c489b6fed8a7/cni-plugins/0.log" Sep 15 07:12:49.481315 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:49.481290 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hddd_178866a3-7693-4a4b-8a07-c489b6fed8a7/bond-cni-plugin/0.log" Sep 15 07:12:49.504075 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:49.504050 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hddd_178866a3-7693-4a4b-8a07-c489b6fed8a7/routeoverride-cni/0.log" Sep 15 07:12:49.530785 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:49.530764 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hddd_178866a3-7693-4a4b-8a07-c489b6fed8a7/whereabouts-cni-bincopy/0.log" Sep 15 07:12:49.551583 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:49.551561 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6hddd_178866a3-7693-4a4b-8a07-c489b6fed8a7/whereabouts-cni/0.log" Sep 15 07:12:49.932148 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:49.932120 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qxgjj_e48e916e-5d40-41bc-8359-bc216fd5b2dd/kube-multus/0.log" Sep 15 07:12:50.004959 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:50.004935 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ns6cw_532a12f1-78ac-4e4d-a02d-6be6c808825f/network-metrics-daemon/0.log" Sep 15 07:12:50.032446 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:50.032421 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ns6cw_532a12f1-78ac-4e4d-a02d-6be6c808825f/kube-rbac-proxy/0.log" Sep 15 07:12:51.285694 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:51.285635 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgvr8_4e07529d-1e9e-4024-8870-0400eb65c215/ovn-controller/0.log" Sep 15 07:12:51.303191 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:51.303164 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgvr8_4e07529d-1e9e-4024-8870-0400eb65c215/ovn-acl-logging/0.log" Sep 15 07:12:51.329542 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:51.329518 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgvr8_4e07529d-1e9e-4024-8870-0400eb65c215/kube-rbac-proxy-node/0.log" Sep 15 07:12:51.349729 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:51.349703 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgvr8_4e07529d-1e9e-4024-8870-0400eb65c215/kube-rbac-proxy-ovn-metrics/0.log" Sep 15 07:12:51.367771 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:51.367751 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgvr8_4e07529d-1e9e-4024-8870-0400eb65c215/northd/0.log" Sep 15 07:12:51.414598 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:51.414578 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgvr8_4e07529d-1e9e-4024-8870-0400eb65c215/nbdb/0.log" Sep 15 07:12:51.458572 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:51.458545 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgvr8_4e07529d-1e9e-4024-8870-0400eb65c215/sbdb/0.log" Sep 15 07:12:51.495049 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:51.495025 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgvr8_4e07529d-1e9e-4024-8870-0400eb65c215/ovnkube-controller/0.log" Sep 15 07:12:52.486332 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:52.486300 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-pnhgk_fa313485-4916-4bd9-9d9e-7d1a126a82a2/network-check-target-container/0.log" Sep 15 07:12:53.397346 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:53.397315 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-z87dk_f694d292-e504-410d-924d-170bfeb5971c/iptables-alerter/0.log" Sep 15 07:12:54.004203 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:54.004173 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-kqt7l_dc8cdf5d-8987-446a-ad13-4605b96a3e0d/tuned/0.log" Sep 15 07:12:57.174086 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:57.174052 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-4r69w_c4e5225b-1c04-42e3-93da-e3bef0b45c73/csi-driver/0.log" Sep 15 07:12:57.194467 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:57.194442 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-4r69w_c4e5225b-1c04-42e3-93da-e3bef0b45c73/csi-node-driver-registrar/0.log" Sep 15 07:12:57.216203 ip-10-0-133-232 kubenswrapper[2552]: I0915 07:12:57.216174 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-4r69w_c4e5225b-1c04-42e3-93da-e3bef0b45c73/csi-liveness-probe/0.log"