Aug 05 12:16:49.872732 ip-10-0-143-178 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Aug 05 12:16:49.872743 ip-10-0-143-178 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Aug 05 12:16:49.872750 ip-10-0-143-178 systemd[1]: kubelet.service: Failed with result 'resources'. Aug 05 12:16:49.872973 ip-10-0-143-178 systemd[1]: Failed to start Kubernetes Kubelet. Aug 05 12:16:59.963679 ip-10-0-143-178 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Aug 05 12:16:59.963696 ip-10-0-143-178 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 7b6bac3c392a4601ab1de22d6cd275a1 -- Aug 05 12:18:54.832220 ip-10-0-143-178 systemd[1]: Starting Kubernetes Kubelet... Aug 05 12:18:55.190761 ip-10-0-143-178 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 05 12:18:55.190761 ip-10-0-143-178 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 05 12:18:55.190761 ip-10-0-143-178 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 05 12:18:55.190761 ip-10-0-143-178 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 05 12:18:55.190761 ip-10-0-143-178 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 05 12:18:55.192675 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.192572 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 05 12:18:55.197068 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197054 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 12:18:55.197068 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197068 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197072 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197081 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197085 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197088 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197091 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197094 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197097 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197100 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197103 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197105 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197108 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197111 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197113 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197116 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197119 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197121 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197124 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197126 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197129 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 12:18:55.197166 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197131 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197134 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197137 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197140 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197142 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197146 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197148 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197151 2569 feature_gate.go:328] unrecognized feature gate: Example Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197153 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197163 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197166 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197169 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197171 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197174 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197177 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197180 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197183 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197185 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197188 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197190 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 12:18:55.197625 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197193 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197195 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197198 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197201 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197203 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197205 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197208 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197210 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197215 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197220 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197224 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197227 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197229 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197232 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197236 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197239 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197242 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197245 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197251 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 12:18:55.198119 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197254 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197256 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197259 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197262 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197265 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197267 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197270 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197272 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197275 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197278 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197282 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197285 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197288 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197290 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197293 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197296 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197298 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197325 2569 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197329 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197332 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 12:18:55.198579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197335 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197338 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197342 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197345 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197347 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197350 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197352 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197355 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197360 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197363 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197366 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197369 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197372 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197375 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197377 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197380 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197383 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197386 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.197388 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 12:18:55.199063 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198087 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198093 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198096 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198099 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198102 2569 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198105 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198108 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198111 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198114 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198117 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198120 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198123 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198126 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198128 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198131 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198134 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198136 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198139 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198141 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198144 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 12:18:55.199509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198148 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198151 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198154 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198157 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198159 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198162 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198164 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198167 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198170 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198172 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198175 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198178 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198181 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198184 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198187 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198189 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198192 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198194 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198198 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198201 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 12:18:55.200205 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198204 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198207 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198209 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198212 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198215 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198217 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198222 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198225 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198229 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198232 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198234 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198237 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198241 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198244 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198247 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198250 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198252 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198256 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198260 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198263 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 12:18:55.200870 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198266 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198268 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198271 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198274 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198277 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198281 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198284 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198286 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198289 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198291 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198294 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198297 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198300 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198302 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198306 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198308 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198311 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198314 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198316 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 12:18:55.201392 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198319 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198322 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198324 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198327 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198329 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198333 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198336 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198340 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198343 2569 feature_gate.go:328] unrecognized feature gate: Example Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198346 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198349 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198351 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198354 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198357 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198359 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198362 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198365 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198368 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198371 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 12:18:55.201894 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198373 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198763 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198769 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198773 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198777 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198780 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198783 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198786 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198789 2569 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198792 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198795 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198798 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198801 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198804 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198807 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198810 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198812 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198815 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198819 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 12:18:55.202381 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198822 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198825 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198828 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198830 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198833 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198835 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198838 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198841 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198843 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198846 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198848 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198851 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198854 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198857 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198859 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198862 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198865 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198867 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198871 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198875 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 12:18:55.202879 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198877 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198880 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198883 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198885 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198888 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198890 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198893 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198895 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198898 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198900 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198903 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198907 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198910 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198913 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198916 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198918 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198921 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198924 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198926 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 12:18:55.203373 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198929 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198933 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198937 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198940 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198943 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198946 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198961 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198964 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198966 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198969 2569 feature_gate.go:328] unrecognized feature gate: Example Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198973 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198976 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198978 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198981 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198983 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198986 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198988 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198991 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198994 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198997 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 12:18:55.203826 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.198999 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199002 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199005 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199007 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199011 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199014 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199016 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199019 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199022 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199024 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199028 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199031 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199033 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199036 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199038 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199041 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199044 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199046 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199049 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199052 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 12:18:55.204345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199055 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.199057 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199125 2569 flags.go:64] FLAG: --address="0.0.0.0" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199140 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199158 2569 flags.go:64] FLAG: --anonymous-auth="true" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199169 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199173 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199177 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199181 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199185 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199188 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199192 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199195 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199199 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199202 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199205 2569 flags.go:64] FLAG: --cgroup-root="" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199209 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199212 2569 flags.go:64] FLAG: --client-ca-file="" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199215 2569 flags.go:64] FLAG: --cloud-provider="external" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199218 2569 flags.go:64] FLAG: --cluster-dns="[]" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199222 2569 flags.go:64] FLAG: --cluster-domain="" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199225 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199228 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199231 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 05 12:18:55.204838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199235 2569 flags.go:64] FLAG: --container-log-max-files="5" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199238 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199241 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199244 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199248 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199251 2569 flags.go:64] FLAG: --contention-profiling="false" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199254 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199258 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199261 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199264 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199269 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199272 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199275 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199278 2569 flags.go:64] FLAG: --enable-load-reader="false" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199281 2569 flags.go:64] FLAG: --enable-server="true" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199285 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199289 2569 flags.go:64] FLAG: --event-burst="100" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199292 2569 flags.go:64] FLAG: --event-qps="50" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199295 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199298 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199301 2569 flags.go:64] FLAG: --eviction-hard="" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199305 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199308 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199311 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199315 2569 flags.go:64] FLAG: --eviction-soft="" Aug 05 12:18:55.205440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199319 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199322 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199324 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199327 2569 flags.go:64] FLAG: --experimental-mounter-path="" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199330 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199333 2569 flags.go:64] FLAG: --fail-swap-on="true" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199336 2569 flags.go:64] FLAG: --feature-gates="" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199340 2569 flags.go:64] FLAG: --file-check-frequency="20s" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199343 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199346 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199349 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199352 2569 flags.go:64] FLAG: --healthz-port="10248" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199355 2569 flags.go:64] FLAG: --help="false" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199358 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199361 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199364 2569 flags.go:64] FLAG: --http-check-frequency="20s" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199367 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199372 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199376 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199379 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199382 2569 flags.go:64] FLAG: --image-service-endpoint="" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199385 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199388 2569 flags.go:64] FLAG: --kube-api-burst="100" Aug 05 12:18:55.206039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199391 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199394 2569 flags.go:64] FLAG: --kube-api-qps="50" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199397 2569 flags.go:64] FLAG: --kube-reserved="" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199400 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199403 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199406 2569 flags.go:64] FLAG: --kubelet-cgroups="" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199409 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199412 2569 flags.go:64] FLAG: --lock-file="" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199416 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199419 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199422 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199427 2569 flags.go:64] FLAG: --log-json-split-stream="false" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199430 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199433 2569 flags.go:64] FLAG: --log-text-split-stream="false" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199436 2569 flags.go:64] FLAG: --logging-format="text" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199439 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199443 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199445 2569 flags.go:64] FLAG: --manifest-url="" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199448 2569 flags.go:64] FLAG: --manifest-url-header="" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199452 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199456 2569 flags.go:64] FLAG: --max-open-files="1000000" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199460 2569 flags.go:64] FLAG: --max-pods="110" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199463 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199466 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199469 2569 flags.go:64] FLAG: --memory-manager-policy="None" Aug 05 12:18:55.206649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199472 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199477 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199480 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199483 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199489 2569 flags.go:64] FLAG: --node-status-max-images="50" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199492 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199495 2569 flags.go:64] FLAG: --oom-score-adj="-999" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199499 2569 flags.go:64] FLAG: --pod-cidr="" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199502 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199507 2569 flags.go:64] FLAG: --pod-manifest-path="" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199510 2569 flags.go:64] FLAG: --pod-max-pids="-1" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199513 2569 flags.go:64] FLAG: --pods-per-core="0" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199516 2569 flags.go:64] FLAG: --port="10250" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199519 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199523 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b10d55d566815019" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199526 2569 flags.go:64] FLAG: --qos-reserved="" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199530 2569 flags.go:64] FLAG: --read-only-port="10255" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199533 2569 flags.go:64] FLAG: --register-node="true" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199536 2569 flags.go:64] FLAG: --register-with-taints="" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199540 2569 flags.go:64] FLAG: --registry-burst="10" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199543 2569 flags.go:64] FLAG: --registry-qps="5" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199546 2569 flags.go:64] FLAG: --reserved-cpus="" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199549 2569 flags.go:64] FLAG: --reserved-memory="" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199553 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199556 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 05 12:18:55.207275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199559 2569 flags.go:64] FLAG: --rotate-certificates="false" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199561 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199564 2569 flags.go:64] FLAG: --runonce="false" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199567 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199570 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199574 2569 flags.go:64] FLAG: --seccomp-default="false" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199576 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199579 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199583 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199587 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199590 2569 flags.go:64] FLAG: --storage-driver-password="root" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199593 2569 flags.go:64] FLAG: --storage-driver-secure="false" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199596 2569 flags.go:64] FLAG: --storage-driver-table="stats" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199599 2569 flags.go:64] FLAG: --storage-driver-user="root" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199602 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199605 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199608 2569 flags.go:64] FLAG: --system-cgroups="" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199610 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199616 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199619 2569 flags.go:64] FLAG: --tls-cert-file="" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199621 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199625 2569 flags.go:64] FLAG: --tls-min-version="" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199628 2569 flags.go:64] FLAG: --tls-private-key-file="" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199634 2569 flags.go:64] FLAG: --topology-manager-policy="none" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199637 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 05 12:18:55.207881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199640 2569 flags.go:64] FLAG: --topology-manager-scope="container" Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199643 2569 flags.go:64] FLAG: --v="2" Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199647 2569 flags.go:64] FLAG: --version="false" Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199651 2569 flags.go:64] FLAG: --vmodule="" Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199655 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.199658 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200158 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200171 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200176 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200181 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200186 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200190 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200194 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200199 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200203 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200208 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200214 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200224 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200229 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200234 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200238 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 12:18:55.208509 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200242 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200246 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200251 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200256 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200260 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200264 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200269 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200273 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200282 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200289 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200293 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200297 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200302 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200306 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200310 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200314 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200318 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200323 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200327 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200331 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 12:18:55.209036 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200336 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200344 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200349 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200353 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200357 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200362 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200374 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200379 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200383 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200388 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200392 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200396 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200400 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200404 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200414 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200418 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200422 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200426 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200430 2569 feature_gate.go:328] unrecognized feature gate: Example Aug 05 12:18:55.209513 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200434 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200441 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200448 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200453 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200458 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200464 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200472 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200482 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200487 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200492 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200499 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200504 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200509 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200513 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200517 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200522 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200526 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200530 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200535 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 12:18:55.209974 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200544 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200548 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200553 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200558 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200563 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200567 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200571 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200576 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200580 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200584 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200588 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200592 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200596 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200606 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200610 2569 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200616 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200620 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200624 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200630 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200635 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 12:18:55.210469 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200639 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200643 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200648 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200658 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200682 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.200706 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.201259 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.208084 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.208097 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.208749 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.208756 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208810 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208815 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208818 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 12:18:55.210976 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208821 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208825 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208828 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208831 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208834 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208837 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208840 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208844 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208847 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208850 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208853 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208856 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208859 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208862 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208864 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208867 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208870 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208872 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208875 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208878 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 12:18:55.211322 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208881 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208884 2569 feature_gate.go:328] unrecognized feature gate: Example Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208886 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208889 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208892 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208895 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208898 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208901 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208903 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208906 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208909 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208911 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208914 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208918 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208922 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208925 2569 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208928 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208931 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208933 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 12:18:55.211795 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208936 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208939 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208943 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208960 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208965 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208969 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208973 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208976 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208978 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208981 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208984 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208987 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208989 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208992 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208995 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.208998 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209001 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209005 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209008 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209012 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 12:18:55.212270 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209014 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209017 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209020 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209023 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209025 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209028 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209030 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209033 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209036 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209038 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209041 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209044 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209047 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209050 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209052 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209055 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209057 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209061 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209063 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 12:18:55.212763 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209066 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209069 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209072 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209075 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209078 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209080 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209083 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209086 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209088 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209091 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209093 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209096 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209099 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209102 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209104 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209107 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209109 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 12:18:55.213221 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209112 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.209117 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209224 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209230 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209232 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209235 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209238 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209240 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209243 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209246 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209249 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209252 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209255 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209257 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 12:18:55.213660 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209260 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209263 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209265 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209268 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209271 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209274 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209276 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209279 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209281 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209284 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209287 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209290 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209292 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209295 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209298 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209301 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209303 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209306 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209309 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209311 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 12:18:55.214022 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209314 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209317 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209319 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209322 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209325 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209327 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209330 2569 feature_gate.go:328] unrecognized feature gate: Example Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209332 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209336 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209339 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209342 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209345 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209347 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209350 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209352 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209355 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209358 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209361 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209363 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209366 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 12:18:55.214496 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209369 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209371 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209373 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209376 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209379 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209381 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209384 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209386 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209388 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209391 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209393 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209396 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209398 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209401 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209404 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209407 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209409 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209413 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209416 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 12:18:55.214980 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209419 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209422 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209425 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209427 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209430 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209432 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209435 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209437 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209440 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209443 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209445 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209450 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209453 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209456 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209459 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209462 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209465 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209467 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209470 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 12:18:55.215433 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209473 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209475 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209478 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209481 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209483 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209486 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209489 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209491 2569 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:55.209495 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.209500 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.209593 2569 server.go:962] "Client rotation is on, will bootstrap in background" Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.211516 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.212516 2569 server.go:1019] "Starting client certificate rotation" Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.212624 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 05 12:18:55.215877 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.213277 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 05 12:18:55.216225 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.214670 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 05 12:18:55.216510 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.216496 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 05 12:18:55.230076 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.229906 2569 log.go:25] "Validated CRI v1 runtime API" Aug 05 12:18:55.232461 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.232446 2569 log.go:25] "Validated CRI v1 image API" Aug 05 12:18:55.235202 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.235180 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 05 12:18:55.237669 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.237650 2569 fs.go:135] Filesystem UUIDs: map[16f91301-1c7a-4336-89dd-abdcd00c6225:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ed7b5bc1-8652-4003-8f90-0fd1a1f99150:/dev/nvme0n1p4] Aug 05 12:18:55.237740 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.237668 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 05 12:18:55.242700 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.242597 2569 manager.go:216] Machine: {Timestamp:2026-08-05 12:18:55.241437573 +0000 UTC m=+0.307029462 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098478 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26ea16e5003654133b957c51ef7191 SystemUUID:ec26ea16-e500-3654-133b-957c51ef7191 BootID:7b6bac3c-392a-4601-ab1d-e22d6cd275a1 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c5:7a:b0:7a:45 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c5:7a:b0:7a:45 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:16:c2:30:25:51 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 05 12:18:55.242700 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.242696 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 05 12:18:55.242806 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.242795 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 05 12:18:55.243587 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.243573 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 05 12:18:55.245909 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.245888 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 05 12:18:55.246073 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.245912 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 05 12:18:55.246663 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.246654 2569 topology_manager.go:138] "Creating topology manager with none policy" Aug 05 12:18:55.246699 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.246666 2569 container_manager_linux.go:309] "Creating device plugin manager" Aug 05 12:18:55.246699 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.246674 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 05 12:18:55.247250 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.247237 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 05 12:18:55.248014 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.248004 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 05 12:18:55.251066 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251055 2569 state_mem.go:36] "Initialized new in-memory state store" Aug 05 12:18:55.251188 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251178 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 05 12:18:55.251226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251209 2569 kubelet.go:486] "Attempting to sync node with API server" Aug 05 12:18:55.251226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251219 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 05 12:18:55.251286 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251229 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 05 12:18:55.251286 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251238 2569 kubelet.go:398] "Adding apiserver pod source" Aug 05 12:18:55.251286 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251246 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 05 12:18:55.251577 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251566 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 05 12:18:55.251639 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.251591 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 05 12:18:55.253647 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.253629 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 05 12:18:55.254896 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.254883 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 05 12:18:55.254969 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.254913 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 05 12:18:55.255865 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255850 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 05 12:18:55.255912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255872 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 05 12:18:55.255912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255882 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 05 12:18:55.255912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255891 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 05 12:18:55.255912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255900 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 05 12:18:55.255912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255906 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 05 12:18:55.255912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255914 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 05 12:18:55.256090 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255920 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 05 12:18:55.256090 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255927 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 05 12:18:55.256090 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255932 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 05 12:18:55.256090 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255945 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 05 12:18:55.256090 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.255968 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 05 12:18:55.256565 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.256555 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 05 12:18:55.256617 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.256573 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 05 12:18:55.259820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.259805 2569 server.go:1268] "Started kubelet" Aug 05 12:18:55.259940 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.259890 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 05 12:18:55.259940 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.259907 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 05 12:18:55.260070 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.259978 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 05 12:18:55.260697 ip-10-0-143-178 systemd[1]: Started Kubernetes Kubelet. Aug 05 12:18:55.262313 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.262292 2569 server.go:310] "Adding debug handlers to kubelet server" Aug 05 12:18:55.264041 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.264014 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 05 12:18:55.264132 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.264103 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 05 12:18:55.264280 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.264250 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 05 12:18:55.264786 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.264761 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 05 12:18:55.268920 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.264897 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-178.ec2.internal.18c8e7ff240af6ad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-178.ec2.internal,UID:ip-10-0-143-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-178.ec2.internal,},FirstTimestamp:2026-08-05 12:18:55.259776685 +0000 UTC m=+0.325368574,LastTimestamp:2026-08-05 12:18:55.259776685 +0000 UTC m=+0.325368574,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-178.ec2.internal,}" Aug 05 12:18:55.269451 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.269432 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 05 12:18:55.271559 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.271545 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 05 12:18:55.271633 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.271558 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 05 12:18:55.271633 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.271620 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:55.272051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.272028 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 05 12:18:55.272372 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.272034 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 05 12:18:55.272372 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.272198 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 05 12:18:55.272372 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.272290 2569 reconstruct.go:97] "Volume reconstruction finished" Aug 05 12:18:55.272372 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.272300 2569 reconciler.go:29] "Reconciler: start to sync state" Aug 05 12:18:55.272696 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.272677 2569 factory.go:55] Registering systemd factory Aug 05 12:18:55.272769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.272758 2569 factory.go:223] Registration of the systemd container factory successfully Aug 05 12:18:55.273030 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.273013 2569 factory.go:153] Registering CRI-O factory Aug 05 12:18:55.273112 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.273033 2569 factory.go:223] Registration of the crio container factory successfully Aug 05 12:18:55.273112 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.273099 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 05 12:18:55.273174 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.273123 2569 factory.go:103] Registering Raw factory Aug 05 12:18:55.273174 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.273138 2569 manager.go:1196] Started watching for new ooms in manager Aug 05 12:18:55.273513 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.273501 2569 manager.go:349] Starting recovery of all containers Aug 05 12:18:55.277991 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.277944 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 05 12:18:55.278111 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.278082 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 05 12:18:55.283668 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.283651 2569 manager.go:354] Recovery completed Aug 05 12:18:55.289034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.289021 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 12:18:55.290711 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.290688 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientMemory" Aug 05 12:18:55.290765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.290723 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 12:18:55.290765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.290738 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientPID" Aug 05 12:18:55.291252 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.291236 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 05 12:18:55.291252 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.291250 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 12:18:55.291364 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.291251 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 05 12:18:55.291364 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.291323 2569 state_mem.go:36] "Initialized new in-memory state store" Aug 05 12:18:55.291928 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.291914 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientMemory" Aug 05 12:18:55.291983 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.291942 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 12:18:55.291983 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.291978 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientPID" Aug 05 12:18:55.292430 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.292418 2569 policy_none.go:49] "None policy: Start" Aug 05 12:18:55.292466 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.292435 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 05 12:18:55.292466 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.292446 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 05 12:18:55.293545 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.293533 2569 policy_none.go:47] "Start" Aug 05 12:18:55.303786 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.303703 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-178.ec2.internal.18c8e7ff25e2fbfe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-178.ec2.internal,UID:ip-10-0-143-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-178.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-178.ec2.internal,},FirstTimestamp:2026-08-05 12:18:55.290711038 +0000 UTC m=+0.356302931,LastTimestamp:2026-08-05 12:18:55.290711038 +0000 UTC m=+0.356302931,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-178.ec2.internal,}" Aug 05 12:18:55.316868 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.316720 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-178.ec2.internal.18c8e7ff25e34632 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-178.ec2.internal,UID:ip-10-0-143-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-178.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-178.ec2.internal,},FirstTimestamp:2026-08-05 12:18:55.290730034 +0000 UTC m=+0.356321925,LastTimestamp:2026-08-05 12:18:55.290730034 +0000 UTC m=+0.356321925,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-178.ec2.internal,}" Aug 05 12:18:55.325474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.325460 2569 manager.go:337] "Starting Device Plugin manager" Aug 05 12:18:55.325547 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.325491 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 05 12:18:55.325547 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.325500 2569 server.go:87] "Starting device plugin registration server" Aug 05 12:18:55.325739 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.325727 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 05 12:18:55.325805 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.325742 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 05 12:18:55.325882 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.325868 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 05 12:18:55.325971 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.325944 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 05 12:18:55.326025 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.325973 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 05 12:18:55.326380 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.326354 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 05 12:18:55.326463 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.326412 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:55.328614 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.328551 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-178.ec2.internal.18c8e7ff25e37b7d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-178.ec2.internal,UID:ip-10-0-143-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-178.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-178.ec2.internal,},FirstTimestamp:2026-08-05 12:18:55.290743677 +0000 UTC m=+0.356335565,LastTimestamp:2026-08-05 12:18:55.290743677 +0000 UTC m=+0.356335565,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-178.ec2.internal,}" Aug 05 12:18:55.343475 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.343455 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8nmv4" Aug 05 12:18:55.344032 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.343974 2569 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-178.ec2.internal.18c8e7ff25e2fbfe\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-178.ec2.internal.18c8e7ff25e2fbfe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-178.ec2.internal,UID:ip-10-0-143-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-178.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-178.ec2.internal,},FirstTimestamp:2026-08-05 12:18:55.290711038 +0000 UTC m=+0.356302931,LastTimestamp:2026-08-05 12:18:55.291928592 +0000 UTC m=+0.357520480,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-178.ec2.internal,}" Aug 05 12:18:55.355192 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.355168 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8nmv4" Aug 05 12:18:55.355430 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.355372 2569 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-178.ec2.internal.18c8e7ff25e34632\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-178.ec2.internal.18c8e7ff25e34632 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-178.ec2.internal,UID:ip-10-0-143-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-178.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-178.ec2.internal,},FirstTimestamp:2026-08-05 12:18:55.290730034 +0000 UTC m=+0.356321925,LastTimestamp:2026-08-05 12:18:55.291970453 +0000 UTC m=+0.357562344,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-178.ec2.internal,}" Aug 05 12:18:55.428084 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.428062 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 12:18:55.428882 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.428865 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientMemory" Aug 05 12:18:55.428954 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.428893 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 12:18:55.428954 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.428904 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientPID" Aug 05 12:18:55.428954 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.428930 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.434440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.434412 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 05 12:18:55.435738 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.435715 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 05 12:18:55.435817 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.435753 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 05 12:18:55.435817 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.435774 2569 kubelet.go:2448] "Starting kubelet main sync loop" Aug 05 12:18:55.435817 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.435808 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 05 12:18:55.438439 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.438423 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 12:18:55.446103 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.446067 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.446103 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.446091 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-178.ec2.internal\": node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:55.497495 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.497474 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:55.538666 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.538643 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal"] Aug 05 12:18:55.538723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.538713 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 12:18:55.540147 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.540133 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientMemory" Aug 05 12:18:55.540217 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.540157 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 12:18:55.540217 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.540174 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientPID" Aug 05 12:18:55.541443 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.541431 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 12:18:55.541559 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.541546 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.541612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.541581 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 12:18:55.542108 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.542094 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientMemory" Aug 05 12:18:55.542173 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.542116 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 12:18:55.542173 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.542127 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientPID" Aug 05 12:18:55.542173 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.542150 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientMemory" Aug 05 12:18:55.542173 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.542169 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 12:18:55.542314 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.542184 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientPID" Aug 05 12:18:55.543334 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.543321 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.543382 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.543348 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 12:18:55.543967 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.543949 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientMemory" Aug 05 12:18:55.544063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.543976 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 12:18:55.544063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.543986 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientPID" Aug 05 12:18:55.566782 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.566758 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-178.ec2.internal\" not found" node="ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.571537 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.571521 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-178.ec2.internal\" not found" node="ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.574838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.574821 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2de2a3e1acdc244afa40a51cd31b1009-config\") pod \"kube-apiserver-proxy-ip-10-0-143-178.ec2.internal\" (UID: \"2de2a3e1acdc244afa40a51cd31b1009\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.574896 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.574847 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d7d621a93826c1936883cd721a37b95a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal\" (UID: \"d7d621a93826c1936883cd721a37b95a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.574896 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.574865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d621a93826c1936883cd721a37b95a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal\" (UID: \"d7d621a93826c1936883cd721a37b95a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.597923 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.597904 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:55.675102 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.675077 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d7d621a93826c1936883cd721a37b95a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal\" (UID: \"d7d621a93826c1936883cd721a37b95a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.675171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.675106 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d621a93826c1936883cd721a37b95a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal\" (UID: \"d7d621a93826c1936883cd721a37b95a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.675171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.675122 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2de2a3e1acdc244afa40a51cd31b1009-config\") pod \"kube-apiserver-proxy-ip-10-0-143-178.ec2.internal\" (UID: \"2de2a3e1acdc244afa40a51cd31b1009\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.675246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.675171 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2de2a3e1acdc244afa40a51cd31b1009-config\") pod \"kube-apiserver-proxy-ip-10-0-143-178.ec2.internal\" (UID: \"2de2a3e1acdc244afa40a51cd31b1009\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.675246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.675180 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d7d621a93826c1936883cd721a37b95a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal\" (UID: \"d7d621a93826c1936883cd721a37b95a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.675246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.675191 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7d621a93826c1936883cd721a37b95a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal\" (UID: \"d7d621a93826c1936883cd721a37b95a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.698480 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.698427 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:55.799691 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.799659 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:55.872429 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.872405 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.874794 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:55.874779 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" Aug 05 12:18:55.899802 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:55.899774 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:56.000970 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:56.000895 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:56.101403 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:56.101376 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:56.202440 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:56.202415 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:56.212314 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.212292 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 05 12:18:56.212497 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.212457 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 05 12:18:56.303241 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:56.303188 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:56.360481 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.360450 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-04 12:13:55 +0000 UTC" deadline="2028-02-18 05:23:03.465252686 +0000 UTC" Aug 05 12:18:56.360481 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.360479 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13481h4m7.104780528s" Aug 05 12:18:56.404127 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:56.404106 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:56.415391 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.415372 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 12:18:56.417259 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.417237 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientMemory" Aug 05 12:18:56.417375 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.417273 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 12:18:56.417375 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.417283 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeHasSufficientPID" Aug 05 12:18:56.460043 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.460005 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 12:18:56.505089 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:56.505065 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-178.ec2.internal\" not found" Aug 05 12:18:56.520236 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:56.520208 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de2a3e1acdc244afa40a51cd31b1009.slice/crio-7ce46b4b67cf781d6c310956ee1c783c5976c2bd5926d8380d59744ef3e29061 WatchSource:0}: Error finding container 7ce46b4b67cf781d6c310956ee1c783c5976c2bd5926d8380d59744ef3e29061: Status 404 returned error can't find the container with id 7ce46b4b67cf781d6c310956ee1c783c5976c2bd5926d8380d59744ef3e29061 Aug 05 12:18:56.520741 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:56.520724 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d621a93826c1936883cd721a37b95a.slice/crio-418aa4e3eb8770a66764a0bbcbd39e1c10de31202ce8d1a0a79d82dc52269d20 WatchSource:0}: Error finding container 418aa4e3eb8770a66764a0bbcbd39e1c10de31202ce8d1a0a79d82dc52269d20: Status 404 returned error can't find the container with id 418aa4e3eb8770a66764a0bbcbd39e1c10de31202ce8d1a0a79d82dc52269d20 Aug 05 12:18:56.525480 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.525457 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 12:18:56.550759 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.550730 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 12:18:56.604123 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.604074 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 12:18:56.675159 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.675136 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" Aug 05 12:18:56.689845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.689828 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 05 12:18:56.690794 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.690780 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" Aug 05 12:18:56.704512 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:56.704494 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 05 12:18:57.254629 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.254597 2569 apiserver.go:52] "Watching apiserver" Aug 05 12:18:57.262238 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.262212 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 05 12:18:57.264324 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.264290 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-b27jc","kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal","openshift-dns/node-resolver-j7c8w","openshift-image-registry/node-ca-p286z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal","openshift-multus/multus-additional-cni-plugins-w4ll9","openshift-multus/network-metrics-daemon-vsh45","openshift-network-diagnostics/network-check-target-z29nk","openshift-ovn-kubernetes/ovnkube-node-k2j42","kube-system/global-pull-secret-syncer-c6j9w","kube-system/konnectivity-agent-2fjj5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s","openshift-cluster-node-tuning-operator/tuned-gg2w9","openshift-multus/multus-tp9z5"] Aug 05 12:18:57.267717 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.266384 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.268263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.267930 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:18:57.268263 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.268031 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:18:57.269060 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.269037 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.269238 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.269213 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2w2n4\"" Aug 05 12:18:57.269341 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.269283 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 05 12:18:57.269455 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.269439 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 05 12:18:57.269549 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.269532 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 05 12:18:57.269616 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.269576 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 05 12:18:57.269827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.269806 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 05 12:18:57.271002 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.270969 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.272268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.272044 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 05 12:18:57.272268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.272224 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 05 12:18:57.272853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.272835 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:57.272950 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.272897 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:18:57.273684 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.273462 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.273684 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.273476 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 05 12:18:57.273829 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.273726 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t9g26\"" Aug 05 12:18:57.273829 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.273807 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-58wp4\"" Aug 05 12:18:57.274077 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.274004 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 05 12:18:57.274178 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.274160 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 05 12:18:57.274412 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.274238 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 05 12:18:57.276025 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.274905 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.276430 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.276409 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.276513 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.276481 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:18:57.278853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.277831 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:18:57.281065 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.280393 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 05 12:18:57.281065 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.280422 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 05 12:18:57.281065 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.280916 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 05 12:18:57.281065 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.280984 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 05 12:18:57.281351 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.281192 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 05 12:18:57.281351 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.281289 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 05 12:18:57.283096 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.281884 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.283096 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.281902 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-256lv\"" Aug 05 12:18:57.283096 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.282297 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 05 12:18:57.283096 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.282398 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 05 12:18:57.285051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.283851 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jfmjl\"" Aug 05 12:18:57.285051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.283952 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 05 12:18:57.285051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.284201 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 05 12:18:57.285051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.284246 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 05 12:18:57.285051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.284204 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bsdkp\"" Aug 05 12:18:57.285358 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.285234 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-knb6m\"" Aug 05 12:18:57.285358 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.285307 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 05 12:18:57.285929 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.285523 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 05 12:18:57.286046 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.285931 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4fhs\" (UniqueName: \"kubernetes.io/projected/00e242a0-2726-4da2-b46e-044a68d05c5d-kube-api-access-q4fhs\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.286046 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.285972 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fh25\" (UniqueName: \"kubernetes.io/projected/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-kube-api-access-9fh25\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.286046 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286005 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:57.286222 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286056 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-system-cni-dir\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.286222 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286183 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59gnl\" (UniqueName: \"kubernetes.io/projected/73786aea-c106-4636-9696-a5ec46bfeb24-kube-api-access-59gnl\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.286330 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286229 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-iptables-alerter-script\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.286330 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286263 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-host\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.286330 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286296 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-cni-binary-copy\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.286330 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286326 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-host-slash\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.286549 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286369 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr444\" (UniqueName: \"kubernetes.io/projected/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-kube-api-access-hr444\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:57.286549 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286506 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-serviceca\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.286653 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286593 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwwc6\" (UniqueName: \"kubernetes.io/projected/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-kube-api-access-qwwc6\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.286708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286664 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-cnibin\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.286708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286699 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-os-release\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.286820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286730 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:18:57.286820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286770 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.286820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286802 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.286970 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286835 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.286970 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286884 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00e242a0-2726-4da2-b46e-044a68d05c5d-hosts-file\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.286970 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.286933 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00e242a0-2726-4da2-b46e-044a68d05c5d-tmp-dir\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.288156 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.287293 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.288156 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.287653 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 05 12:18:57.289679 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.288894 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.289755 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.289698 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 05 12:18:57.292222 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.292205 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9zm96\"" Aug 05 12:18:57.292812 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.292500 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-z5c6t\"" Aug 05 12:18:57.293599 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.292553 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 05 12:18:57.293873 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.293355 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 05 12:18:57.294172 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.293414 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 05 12:18:57.312655 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.312625 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qxxhb" Aug 05 12:18:57.321079 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.321017 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qxxhb" Aug 05 12:18:57.365307 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.365284 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 12:18:57.373355 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.373337 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 05 12:18:57.387733 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387703 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-conf-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.387845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387746 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-slash\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.387845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387772 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-run-netns\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.387845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387799 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-sys-fs\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.387845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387838 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-host-slash\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.388068 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-daemon-config\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.388068 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387890 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-var-lib-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.388068 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387914 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-cni-netd\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.388068 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387939 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-ovnkube-script-lib\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.388068 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.387964 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b6fm\" (UniqueName: \"kubernetes.io/projected/6663f904-8a23-4be1-9d95-6c7b4585a35a-kube-api-access-2b6fm\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.388256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388083 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-host-slash\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.388256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388130 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:18:57.388256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388156 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-tuned\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.388256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388181 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-host\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.388256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388205 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-kubernetes\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.388256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388234 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00e242a0-2726-4da2-b46e-044a68d05c5d-tmp-dir\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388258 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-os-release\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388263 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-host\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388284 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-cni-binary-copy\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388321 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4155f61-9591-437e-a077-b395af2f689e-ovn-node-metrics-cert\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388378 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388414 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysctl-d\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388441 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-tmp\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388470 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-system-cni-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388495 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-socket-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388523 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-device-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388523 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00e242a0-2726-4da2-b46e-044a68d05c5d-tmp-dir\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.388550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr444\" (UniqueName: \"kubernetes.io/projected/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-kube-api-access-hr444\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-lib-modules\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388609 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-socket-dir-parent\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388635 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-systemd-units\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388661 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-ovnkube-config\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388688 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-kubelet\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388715 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-cni-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388742 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-netns\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388770 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-cni-bin\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388811 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-etc-kubernetes\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388836 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-etc-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388860 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-log-socket\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388897 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysconfig\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388931 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.388959 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389007 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-hostroot\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389034 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.389170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389061 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf46c\" (UniqueName: \"kubernetes.io/projected/e4155f61-9591-437e-a077-b395af2f689e-kube-api-access-zf46c\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389104 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a352b881-3036-4ed7-971a-22645878ebfd-kubelet-config\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389111 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389129 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389164 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-systemd\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.389213 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-ovn\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389290 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-cni-binary-copy\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.389351 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs podName:bccda255-92ed-4f4c-b52f-f3e7c6a639bf nodeName:}" failed. No retries permitted until 2026-08-05 12:18:57.889316484 +0000 UTC m=+2.954908377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs") pod "network-metrics-daemon-vsh45" (UID: "bccda255-92ed-4f4c-b52f-f3e7c6a639bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389376 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-multus-certs\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389406 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61f27d7c-18bb-4986-98b1-0908ccfeec3e-agent-certs\") pod \"konnectivity-agent-2fjj5\" (UID: \"61f27d7c-18bb-4986-98b1-0908ccfeec3e\") " pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389585 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysctl-conf\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389613 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-run\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389654 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-sys\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389697 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-k8s-cni-cncf-io\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389726 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-cni-bin\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.390109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389751 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-env-overrides\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389780 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389810 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00e242a0-2726-4da2-b46e-044a68d05c5d-hosts-file\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389833 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-var-lib-kubelet\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389857 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-host\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389855 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-cni-binary-copy\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389937 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-cni-multus\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.389965 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-node-log\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390030 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/00e242a0-2726-4da2-b46e-044a68d05c5d-hosts-file\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390030 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a352b881-3036-4ed7-971a-22645878ebfd-dbus\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390072 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-system-cni-dir\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390101 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb8bs\" (UniqueName: \"kubernetes.io/projected/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-kube-api-access-bb8bs\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390127 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-system-cni-dir\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390176 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390208 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-serviceca\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390246 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwwc6\" (UniqueName: \"kubernetes.io/projected/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-kube-api-access-qwwc6\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390273 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-modprobe-d\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.390918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390355 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390393 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61f27d7c-18bb-4986-98b1-0908ccfeec3e-konnectivity-ca\") pod \"konnectivity-agent-2fjj5\" (UID: \"61f27d7c-18bb-4986-98b1-0908ccfeec3e\") " pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390450 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-etc-selinux\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390490 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-cnibin\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390529 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-os-release\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390570 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xn27\" (UniqueName: \"kubernetes.io/projected/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-kube-api-access-4xn27\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390590 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-os-release\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390607 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-cnibin\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390641 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73786aea-c106-4636-9696-a5ec46bfeb24-cnibin\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390680 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-run-ovn-kubernetes\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390709 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390744 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4fhs\" (UniqueName: \"kubernetes.io/projected/00e242a0-2726-4da2-b46e-044a68d05c5d-kube-api-access-q4fhs\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390759 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-serviceca\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390825 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fh25\" (UniqueName: \"kubernetes.io/projected/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-kube-api-access-9fh25\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-registration-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59gnl\" (UniqueName: \"kubernetes.io/projected/73786aea-c106-4636-9696-a5ec46bfeb24-kube-api-access-59gnl\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.391552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390926 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-iptables-alerter-script\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.392107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390955 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-systemd\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.392107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.390998 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-kubelet\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.392107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.391314 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73786aea-c106-4636-9696-a5ec46bfeb24-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.392107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.392002 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-iptables-alerter-script\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.395715 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.395693 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 12:18:57.395811 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.395720 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 12:18:57.395811 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.395732 2569 projected.go:196] Error preparing data for projected volume kube-api-access-prb2h for pod openshift-network-diagnostics/network-check-target-z29nk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:18:57.395811 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.395780 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h podName:bb6cf0be-ed34-4473-b64d-d12bf3212eae nodeName:}" failed. No retries permitted until 2026-08-05 12:18:57.895763372 +0000 UTC m=+2.961355264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-prb2h" (UniqueName: "kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h") pod "network-check-target-z29nk" (UID: "bb6cf0be-ed34-4473-b64d-d12bf3212eae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:18:57.396260 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.396228 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 05 12:18:57.400542 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.400519 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr444\" (UniqueName: \"kubernetes.io/projected/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-kube-api-access-hr444\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:57.401262 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.401239 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4fhs\" (UniqueName: \"kubernetes.io/projected/00e242a0-2726-4da2-b46e-044a68d05c5d-kube-api-access-q4fhs\") pod \"node-resolver-j7c8w\" (UID: \"00e242a0-2726-4da2-b46e-044a68d05c5d\") " pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.402238 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.402188 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fh25\" (UniqueName: \"kubernetes.io/projected/d3391f80-10ec-4a2c-abe5-a50d4cd2bc24-kube-api-access-9fh25\") pod \"iptables-alerter-b27jc\" (UID: \"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24\") " pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.402386 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.402367 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwwc6\" (UniqueName: \"kubernetes.io/projected/b8d4ced4-b21e-4a12-8a35-269ad385fcc6-kube-api-access-qwwc6\") pod \"node-ca-p286z\" (UID: \"b8d4ced4-b21e-4a12-8a35-269ad385fcc6\") " pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.404146 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.404125 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59gnl\" (UniqueName: \"kubernetes.io/projected/73786aea-c106-4636-9696-a5ec46bfeb24-kube-api-access-59gnl\") pod \"multus-additional-cni-plugins-w4ll9\" (UID: \"73786aea-c106-4636-9696-a5ec46bfeb24\") " pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.441596 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.441547 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" event={"ID":"2de2a3e1acdc244afa40a51cd31b1009","Type":"ContainerStarted","Data":"7ce46b4b67cf781d6c310956ee1c783c5976c2bd5926d8380d59744ef3e29061"} Aug 05 12:18:57.443158 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.443132 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" event={"ID":"d7d621a93826c1936883cd721a37b95a","Type":"ContainerStarted","Data":"418aa4e3eb8770a66764a0bbcbd39e1c10de31202ce8d1a0a79d82dc52269d20"} Aug 05 12:18:57.492058 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492027 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-systemd\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.492187 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492064 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-ovn\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.492187 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492086 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-multus-certs\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.492187 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492103 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61f27d7c-18bb-4986-98b1-0908ccfeec3e-agent-certs\") pod \"konnectivity-agent-2fjj5\" (UID: \"61f27d7c-18bb-4986-98b1-0908ccfeec3e\") " pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:18:57.492187 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492132 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysctl-conf\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.492187 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492151 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-run\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.492187 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492172 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-sys\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492194 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-k8s-cni-cncf-io\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492211 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-cni-bin\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492226 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-env-overrides\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492249 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-var-lib-kubelet\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492266 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-host\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492281 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-cni-multus\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492296 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-node-log\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492328 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a352b881-3036-4ed7-971a-22645878ebfd-dbus\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492351 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb8bs\" (UniqueName: \"kubernetes.io/projected/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-kube-api-access-bb8bs\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492369 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492389 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-modprobe-d\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492404 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61f27d7c-18bb-4986-98b1-0908ccfeec3e-konnectivity-ca\") pod \"konnectivity-agent-2fjj5\" (UID: \"61f27d7c-18bb-4986-98b1-0908ccfeec3e\") " pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492427 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-etc-selinux\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492454 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xn27\" (UniqueName: \"kubernetes.io/projected/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-kube-api-access-4xn27\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492474 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-cnibin\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.492500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492496 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-run-ovn-kubernetes\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492523 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-registration-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492541 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-systemd\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492556 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-kubelet\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492580 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-conf-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492605 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-slash\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492630 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-run-netns\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-sys-fs\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492684 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-daemon-config\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492708 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-var-lib-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492743 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-cni-netd\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492769 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-ovnkube-script-lib\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b6fm\" (UniqueName: \"kubernetes.io/projected/6663f904-8a23-4be1-9d95-6c7b4585a35a-kube-api-access-2b6fm\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492838 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-tuned\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492864 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-kubernetes\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492891 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-os-release\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-cni-binary-copy\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.493237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492948 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4155f61-9591-437e-a077-b395af2f689e-ovn-node-metrics-cert\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.492974 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493000 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysctl-d\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493025 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-tmp\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493068 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-system-cni-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493092 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-socket-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493118 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-device-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493146 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-lib-modules\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493173 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-socket-dir-parent\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493200 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-systemd-units\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493226 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-ovnkube-config\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493254 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-kubelet\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493281 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-cni-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493299 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-netns\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493315 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-cni-bin\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493341 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-etc-kubernetes\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493359 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-etc-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493404 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-log-socket\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.494107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493434 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysconfig\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493477 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-hostroot\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493478 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-slash\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493511 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493564 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493566 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf46c\" (UniqueName: \"kubernetes.io/projected/e4155f61-9591-437e-a077-b395af2f689e-kube-api-access-zf46c\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493613 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a352b881-3036-4ed7-971a-22645878ebfd-kubelet-config\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493645 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493809 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493875 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-ovn\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493911 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-systemd\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493930 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-multus-certs\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.493952 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-run-netns\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.495023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.494011 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-sys-fs\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.502774 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.502749 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-sys\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.502774 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.502759 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-cni-multus\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.502933 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.502850 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-var-lib-kubelet\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.502933 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.502876 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-node-log\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.503076 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.502944 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-host\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.503076 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503052 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-var-lib-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.503184 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503161 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-modprobe-d\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.503184 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503174 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-systemd\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.503297 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503175 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-daemon-config\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.503297 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503237 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-kubelet\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.503409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503305 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-env-overrides\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.503409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503312 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-run-ovn-kubernetes\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.503409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503337 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-run\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.503409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503382 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysctl-conf\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.503615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503414 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-registration-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.503615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503410 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-etc-selinux\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.503615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503517 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-cni-bin\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.503615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503581 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-conf-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.503827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503614 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-k8s-cni-cncf-io\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.503827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503670 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-cnibin\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.503827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503738 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-kubernetes\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.503827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503385 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-cni-netd\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.504055 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503825 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-systemd-units\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.504055 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.503914 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-os-release\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.504174 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.504060 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61f27d7c-18bb-4986-98b1-0908ccfeec3e-konnectivity-ca\") pod \"konnectivity-agent-2fjj5\" (UID: \"61f27d7c-18bb-4986-98b1-0908ccfeec3e\") " pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:18:57.504290 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.504203 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-ovnkube-script-lib\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.504504 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.504483 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-system-cni-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.504941 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.504842 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a352b881-3036-4ed7-971a-22645878ebfd-dbus\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.505387 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505366 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-socket-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.505476 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505434 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6663f904-8a23-4be1-9d95-6c7b4585a35a-device-dir\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.505589 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505545 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-lib-modules\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.505589 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505563 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-cni-binary-copy\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.505702 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505607 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-socket-dir-parent\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.505702 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505640 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-etc-kubernetes\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.505802 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505712 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysctl-d\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.505863 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.505799 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 12:18:57.505863 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505805 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-multus-cni-dir\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.505863 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.505846 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-host-kubelet\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.505863 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.505856 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret podName:a352b881-3036-4ed7-971a-22645878ebfd nodeName:}" failed. No retries permitted until 2026-08-05 12:18:58.00583745 +0000 UTC m=+3.071429334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret") pod "global-pull-secret-syncer-c6j9w" (UID: "a352b881-3036-4ed7-971a-22645878ebfd") : object "kube-system"/"original-pull-secret" not registered Aug 05 12:18:57.506153 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506095 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-run-netns\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.506153 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506134 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e4155f61-9591-437e-a077-b395af2f689e-ovnkube-config\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.506276 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506153 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-host-var-lib-cni-bin\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.506276 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506164 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-sysconfig\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.506276 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506170 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-log-socket\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.506276 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506200 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-etc-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.506276 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506224 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-hostroot\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.506276 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506229 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a352b881-3036-4ed7-971a-22645878ebfd-kubelet-config\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:57.506276 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.506242 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e4155f61-9591-437e-a077-b395af2f689e-run-openvswitch\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.507248 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.507012 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf46c\" (UniqueName: \"kubernetes.io/projected/e4155f61-9591-437e-a077-b395af2f689e-kube-api-access-zf46c\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.507766 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.507747 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-etc-tuned\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.509010 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.508986 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-tmp\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.509656 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.509410 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61f27d7c-18bb-4986-98b1-0908ccfeec3e-agent-certs\") pod \"konnectivity-agent-2fjj5\" (UID: \"61f27d7c-18bb-4986-98b1-0908ccfeec3e\") " pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:18:57.511166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.511142 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e4155f61-9591-437e-a077-b395af2f689e-ovn-node-metrics-cert\") pod \"ovnkube-node-k2j42\" (UID: \"e4155f61-9591-437e-a077-b395af2f689e\") " pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.511967 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.511924 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb8bs\" (UniqueName: \"kubernetes.io/projected/d6a8082e-295e-4dc6-ae1e-35ce58a50e8f-kube-api-access-bb8bs\") pod \"multus-tp9z5\" (UID: \"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f\") " pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.515304 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.515247 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b6fm\" (UniqueName: \"kubernetes.io/projected/6663f904-8a23-4be1-9d95-6c7b4585a35a-kube-api-access-2b6fm\") pod \"aws-ebs-csi-driver-node-www5s\" (UID: \"6663f904-8a23-4be1-9d95-6c7b4585a35a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.515865 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.515816 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xn27\" (UniqueName: \"kubernetes.io/projected/b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d-kube-api-access-4xn27\") pod \"tuned-gg2w9\" (UID: \"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d\") " pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.584117 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.584084 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" Aug 05 12:18:57.593591 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.593564 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j7c8w" Aug 05 12:18:57.607589 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.607563 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b27jc" Aug 05 12:18:57.616590 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.616572 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p286z" Aug 05 12:18:57.625601 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.625575 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:18:57.637456 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.637424 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:18:57.645105 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.645086 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" Aug 05 12:18:57.653841 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.653819 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" Aug 05 12:18:57.661627 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.661608 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tp9z5" Aug 05 12:18:57.896815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.896733 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:57.896982 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:57.896840 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:18:57.896982 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.896910 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:18:57.897098 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.896999 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs podName:bccda255-92ed-4f4c-b52f-f3e7c6a639bf nodeName:}" failed. No retries permitted until 2026-08-05 12:18:58.896959534 +0000 UTC m=+3.962551657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs") pod "network-metrics-daemon-vsh45" (UID: "bccda255-92ed-4f4c-b52f-f3e7c6a639bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:18:57.897098 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.897015 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 12:18:57.897098 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.897042 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 12:18:57.897098 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.897056 2569 projected.go:196] Error preparing data for projected volume kube-api-access-prb2h for pod openshift-network-diagnostics/network-check-target-z29nk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:18:57.897253 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:57.897103 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h podName:bb6cf0be-ed34-4473-b64d-d12bf3212eae nodeName:}" failed. No retries permitted until 2026-08-05 12:18:58.897090144 +0000 UTC m=+3.962682022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-prb2h" (UniqueName: "kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h") pod "network-check-target-z29nk" (UID: "bb6cf0be-ed34-4473-b64d-d12bf3212eae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:18:58.098217 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:58.098181 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:58.098392 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:58.098297 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 12:18:58.098392 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:58.098358 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret podName:a352b881-3036-4ed7-971a-22645878ebfd nodeName:}" failed. No retries permitted until 2026-08-05 12:18:59.098337442 +0000 UTC m=+4.163929325 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret") pod "global-pull-secret-syncer-c6j9w" (UID: "a352b881-3036-4ed7-971a-22645878ebfd") : object "kube-system"/"original-pull-secret" not registered Aug 05 12:18:58.325040 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:58.324938 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 12:13:57 +0000 UTC" deadline="2028-04-25 21:53:21.219587336 +0000 UTC" Aug 05 12:18:58.325040 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:58.324989 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15105h34m22.894604182s" Aug 05 12:18:58.647877 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.647846 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e242a0_2726_4da2_b46e_044a68d05c5d.slice/crio-7f8f056a941b61ccd8cce0b1dafa90e99c798e6004e343a0116a963faf3a9db0 WatchSource:0}: Error finding container 7f8f056a941b61ccd8cce0b1dafa90e99c798e6004e343a0116a963faf3a9db0: Status 404 returned error can't find the container with id 7f8f056a941b61ccd8cce0b1dafa90e99c798e6004e343a0116a963faf3a9db0 Aug 05 12:18:58.651345 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.651235 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73786aea_c106_4636_9696_a5ec46bfeb24.slice/crio-90b3f70b5afc513bf1483e611ea74e6b4147f903d1453705de9ab3024a37e0d6 WatchSource:0}: Error finding container 90b3f70b5afc513bf1483e611ea74e6b4147f903d1453705de9ab3024a37e0d6: Status 404 returned error can't find the container with id 90b3f70b5afc513bf1483e611ea74e6b4147f903d1453705de9ab3024a37e0d6 Aug 05 12:18:58.653397 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.653376 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6663f904_8a23_4be1_9d95_6c7b4585a35a.slice/crio-cdcc717514e12828b04c2de03964bd81e9edd90e9ff771f6fe0c5c124d1cbe94 WatchSource:0}: Error finding container cdcc717514e12828b04c2de03964bd81e9edd90e9ff771f6fe0c5c124d1cbe94: Status 404 returned error can't find the container with id cdcc717514e12828b04c2de03964bd81e9edd90e9ff771f6fe0c5c124d1cbe94 Aug 05 12:18:58.654789 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.654760 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3391f80_10ec_4a2c_abe5_a50d4cd2bc24.slice/crio-02484a6a19b8a2dc3f17a3c3d229898ebff899d42e913979fa286204fe75972e WatchSource:0}: Error finding container 02484a6a19b8a2dc3f17a3c3d229898ebff899d42e913979fa286204fe75972e: Status 404 returned error can't find the container with id 02484a6a19b8a2dc3f17a3c3d229898ebff899d42e913979fa286204fe75972e Aug 05 12:18:58.656151 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.656127 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb01b0a6a_4ac3_48f9_aa3d_2ab421bdbd9d.slice/crio-1ed0f63168f13a49a0ced59d1156908466230cb3a7d2a8eaea3232851738ad43 WatchSource:0}: Error finding container 1ed0f63168f13a49a0ced59d1156908466230cb3a7d2a8eaea3232851738ad43: Status 404 returned error can't find the container with id 1ed0f63168f13a49a0ced59d1156908466230cb3a7d2a8eaea3232851738ad43 Aug 05 12:18:58.656636 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.656616 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a8082e_295e_4dc6_ae1e_35ce58a50e8f.slice/crio-f4b1edcfccbf17b3cff1a1f966d141e87f8085a870cc6c9ce1e97ded01826d01 WatchSource:0}: Error finding container f4b1edcfccbf17b3cff1a1f966d141e87f8085a870cc6c9ce1e97ded01826d01: Status 404 returned error can't find the container with id f4b1edcfccbf17b3cff1a1f966d141e87f8085a870cc6c9ce1e97ded01826d01 Aug 05 12:18:58.658084 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.657889 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d4ced4_b21e_4a12_8a35_269ad385fcc6.slice/crio-68a815130cd7952ccbabc6a0689d0d0779cb5e7ec82266575943fc5ec71a914b WatchSource:0}: Error finding container 68a815130cd7952ccbabc6a0689d0d0779cb5e7ec82266575943fc5ec71a914b: Status 404 returned error can't find the container with id 68a815130cd7952ccbabc6a0689d0d0779cb5e7ec82266575943fc5ec71a914b Aug 05 12:18:58.659447 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.659423 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4155f61_9591_437e_a077_b395af2f689e.slice/crio-0639f49906b59cfc2838e3b1e3da5aeccaf892a17d90fc390d52771caa0d531e WatchSource:0}: Error finding container 0639f49906b59cfc2838e3b1e3da5aeccaf892a17d90fc390d52771caa0d531e: Status 404 returned error can't find the container with id 0639f49906b59cfc2838e3b1e3da5aeccaf892a17d90fc390d52771caa0d531e Aug 05 12:18:58.663944 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:18:58.661987 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f27d7c_18bb_4986_98b1_0908ccfeec3e.slice/crio-57b1a49de69ec2a73f462c4b29c849f5def3084d30895fc329b6653a987b321a WatchSource:0}: Error finding container 57b1a49de69ec2a73f462c4b29c849f5def3084d30895fc329b6653a987b321a: Status 404 returned error can't find the container with id 57b1a49de69ec2a73f462c4b29c849f5def3084d30895fc329b6653a987b321a Aug 05 12:18:58.904260 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:58.904078 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:18:58.904348 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:58.904231 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 12:18:58.904399 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:58.904360 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 12:18:58.904399 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:58.904372 2569 projected.go:196] Error preparing data for projected volume kube-api-access-prb2h for pod openshift-network-diagnostics/network-check-target-z29nk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:18:58.904476 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:58.904414 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h podName:bb6cf0be-ed34-4473-b64d-d12bf3212eae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:00.904397924 +0000 UTC m=+5.969989800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-prb2h" (UniqueName: "kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h") pod "network-check-target-z29nk" (UID: "bb6cf0be-ed34-4473-b64d-d12bf3212eae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:18:58.904476 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:58.904414 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:18:58.904476 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:58.904460 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs podName:bccda255-92ed-4f4c-b52f-f3e7c6a639bf nodeName:}" failed. No retries permitted until 2026-08-05 12:19:00.904448476 +0000 UTC m=+5.970040353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs") pod "network-metrics-daemon-vsh45" (UID: "bccda255-92ed-4f4c-b52f-f3e7c6a639bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:18:58.904476 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:58.904329 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:59.106562 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.106515 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:59.106724 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:59.106691 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 12:18:59.106815 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:59.106771 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret podName:a352b881-3036-4ed7-971a-22645878ebfd nodeName:}" failed. No retries permitted until 2026-08-05 12:19:01.106749457 +0000 UTC m=+6.172341340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret") pod "global-pull-secret-syncer-c6j9w" (UID: "a352b881-3036-4ed7-971a-22645878ebfd") : object "kube-system"/"original-pull-secret" not registered Aug 05 12:18:59.325833 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.325740 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 12:13:57 +0000 UTC" deadline="2028-02-22 11:12:35.383383312 +0000 UTC" Aug 05 12:18:59.325833 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.325786 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13582h53m36.057603983s" Aug 05 12:18:59.437791 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.437085 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:18:59.437791 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:59.437220 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:18:59.437791 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.437653 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:18:59.437791 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:59.437749 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:18:59.438353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.438215 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:18:59.438353 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:18:59.438321 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:18:59.461193 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.461159 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" event={"ID":"6663f904-8a23-4be1-9d95-6c7b4585a35a","Type":"ContainerStarted","Data":"cdcc717514e12828b04c2de03964bd81e9edd90e9ff771f6fe0c5c124d1cbe94"} Aug 05 12:18:59.466166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.466119 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j7c8w" event={"ID":"00e242a0-2726-4da2-b46e-044a68d05c5d","Type":"ContainerStarted","Data":"7f8f056a941b61ccd8cce0b1dafa90e99c798e6004e343a0116a963faf3a9db0"} Aug 05 12:18:59.476459 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.476432 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2fjj5" event={"ID":"61f27d7c-18bb-4986-98b1-0908ccfeec3e","Type":"ContainerStarted","Data":"57b1a49de69ec2a73f462c4b29c849f5def3084d30895fc329b6653a987b321a"} Aug 05 12:18:59.487118 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.487089 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b27jc" event={"ID":"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24","Type":"ContainerStarted","Data":"02484a6a19b8a2dc3f17a3c3d229898ebff899d42e913979fa286204fe75972e"} Aug 05 12:18:59.501879 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.501803 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerStarted","Data":"90b3f70b5afc513bf1483e611ea74e6b4147f903d1453705de9ab3024a37e0d6"} Aug 05 12:18:59.516888 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.516858 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" event={"ID":"2de2a3e1acdc244afa40a51cd31b1009","Type":"ContainerStarted","Data":"a52a39ed713e4ae71f25a33462a0fbc814ddecb8dab803f6936777eb28db5229"} Aug 05 12:18:59.541057 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.538160 2569 generic.go:356] "Generic (PLEG): container finished" podID="d7d621a93826c1936883cd721a37b95a" containerID="87200fbfe0381e94a9f9963c7ba2fa57f639025bb095496f4c4aca48e79f71b3" exitCode=0 Aug 05 12:18:59.541057 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.538302 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" event={"ID":"d7d621a93826c1936883cd721a37b95a","Type":"ContainerDied","Data":"87200fbfe0381e94a9f9963c7ba2fa57f639025bb095496f4c4aca48e79f71b3"} Aug 05 12:18:59.548038 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.547940 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"0639f49906b59cfc2838e3b1e3da5aeccaf892a17d90fc390d52771caa0d531e"} Aug 05 12:18:59.551050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.550961 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p286z" event={"ID":"b8d4ced4-b21e-4a12-8a35-269ad385fcc6","Type":"ContainerStarted","Data":"68a815130cd7952ccbabc6a0689d0d0779cb5e7ec82266575943fc5ec71a914b"} Aug 05 12:18:59.553755 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.553678 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tp9z5" event={"ID":"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f","Type":"ContainerStarted","Data":"f4b1edcfccbf17b3cff1a1f966d141e87f8085a870cc6c9ce1e97ded01826d01"} Aug 05 12:18:59.555149 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.555087 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" event={"ID":"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d","Type":"ContainerStarted","Data":"1ed0f63168f13a49a0ced59d1156908466230cb3a7d2a8eaea3232851738ad43"} Aug 05 12:18:59.562925 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:18:59.562356 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-178.ec2.internal" podStartSLOduration=2.562338248 podStartE2EDuration="2.562338248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:18:59.534774566 +0000 UTC m=+4.600366471" watchObservedRunningTime="2026-08-05 12:18:59.562338248 +0000 UTC m=+4.627930152" Aug 05 12:19:00.568533 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:00.568494 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" event={"ID":"d7d621a93826c1936883cd721a37b95a","Type":"ContainerStarted","Data":"a8f79bb68025f64c755a6d6da486568ae69340a8f2d9f86769d9b9339dcccdb3"} Aug 05 12:19:00.586330 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:00.586273 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-178.ec2.internal" podStartSLOduration=3.586250174 podStartE2EDuration="3.586250174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:00.585556733 +0000 UTC m=+5.651148636" watchObservedRunningTime="2026-08-05 12:19:00.586250174 +0000 UTC m=+5.651842078" Aug 05 12:19:00.926035 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:00.925938 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:00.926035 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:00.926037 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:00.926260 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:00.926198 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:19:00.926299 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:00.926265 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs podName:bccda255-92ed-4f4c-b52f-f3e7c6a639bf nodeName:}" failed. No retries permitted until 2026-08-05 12:19:04.92624499 +0000 UTC m=+9.991836874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs") pod "network-metrics-daemon-vsh45" (UID: "bccda255-92ed-4f4c-b52f-f3e7c6a639bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:19:00.926860 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:00.926785 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 12:19:00.926860 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:00.926861 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 12:19:00.927022 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:00.926876 2569 projected.go:196] Error preparing data for projected volume kube-api-access-prb2h for pod openshift-network-diagnostics/network-check-target-z29nk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:19:00.927022 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:00.926930 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h podName:bb6cf0be-ed34-4473-b64d-d12bf3212eae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:04.926912684 +0000 UTC m=+9.992504581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-prb2h" (UniqueName: "kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h") pod "network-check-target-z29nk" (UID: "bb6cf0be-ed34-4473-b64d-d12bf3212eae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:19:01.127966 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:01.127919 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:01.128130 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:01.128069 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 12:19:01.128188 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:01.128130 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret podName:a352b881-3036-4ed7-971a-22645878ebfd nodeName:}" failed. No retries permitted until 2026-08-05 12:19:05.128112725 +0000 UTC m=+10.193704606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret") pod "global-pull-secret-syncer-c6j9w" (UID: "a352b881-3036-4ed7-971a-22645878ebfd") : object "kube-system"/"original-pull-secret" not registered Aug 05 12:19:01.439233 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:01.439202 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:01.439404 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:01.439344 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:01.439731 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:01.439700 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:01.439824 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:01.439797 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:01.439883 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:01.439849 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:01.439936 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:01.439921 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:03.436475 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:03.436441 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:03.436938 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:03.436584 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:03.436938 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:03.436694 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:03.436938 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:03.436787 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:03.436938 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:03.436865 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:03.437174 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:03.436943 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:04.961215 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:04.961165 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:04.961664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:04.961247 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:04.961664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:04.961379 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 12:19:04.961664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:04.961395 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:19:04.961664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:04.961404 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 12:19:04.961664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:04.961422 2569 projected.go:196] Error preparing data for projected volume kube-api-access-prb2h for pod openshift-network-diagnostics/network-check-target-z29nk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:19:04.961664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:04.961462 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs podName:bccda255-92ed-4f4c-b52f-f3e7c6a639bf nodeName:}" failed. No retries permitted until 2026-08-05 12:19:12.961442538 +0000 UTC m=+18.027034418 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs") pod "network-metrics-daemon-vsh45" (UID: "bccda255-92ed-4f4c-b52f-f3e7c6a639bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:19:04.961664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:04.961494 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h podName:bb6cf0be-ed34-4473-b64d-d12bf3212eae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:12.9614781 +0000 UTC m=+18.027069979 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-prb2h" (UniqueName: "kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h") pod "network-check-target-z29nk" (UID: "bb6cf0be-ed34-4473-b64d-d12bf3212eae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:19:05.162056 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:05.162019 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:05.162227 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:05.162183 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 12:19:05.162282 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:05.162259 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret podName:a352b881-3036-4ed7-971a-22645878ebfd nodeName:}" failed. No retries permitted until 2026-08-05 12:19:13.162239307 +0000 UTC m=+18.227831190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret") pod "global-pull-secret-syncer-c6j9w" (UID: "a352b881-3036-4ed7-971a-22645878ebfd") : object "kube-system"/"original-pull-secret" not registered Aug 05 12:19:05.437322 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:05.437249 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:05.437491 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:05.437359 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:05.437969 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:05.437701 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:05.437969 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:05.437797 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:05.437969 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:05.437842 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:05.437969 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:05.437918 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:07.436693 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:07.436659 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:07.437198 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:07.436660 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:07.437198 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:07.436791 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:07.437198 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:07.436856 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:07.437198 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:07.436939 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:07.437198 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:07.437031 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:09.436222 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:09.436188 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:09.436664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:09.436188 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:09.436664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:09.436305 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:09.436664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:09.436193 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:09.436664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:09.436361 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:09.436664 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:09.436459 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:11.436592 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:11.436559 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:11.437089 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:11.436558 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:11.437089 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:11.436675 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:11.437089 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:11.436726 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:11.437089 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:11.436811 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:11.437089 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:11.436880 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:13.016858 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:13.016824 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:13.017310 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:13.016900 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:13.017310 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.016968 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:19:13.017310 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.017037 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs podName:bccda255-92ed-4f4c-b52f-f3e7c6a639bf nodeName:}" failed. No retries permitted until 2026-08-05 12:19:29.017019191 +0000 UTC m=+34.082611066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs") pod "network-metrics-daemon-vsh45" (UID: "bccda255-92ed-4f4c-b52f-f3e7c6a639bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:19:13.017310 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.017049 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 12:19:13.017310 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.017069 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 12:19:13.017310 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.017082 2569 projected.go:196] Error preparing data for projected volume kube-api-access-prb2h for pod openshift-network-diagnostics/network-check-target-z29nk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:19:13.017310 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.017133 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h podName:bb6cf0be-ed34-4473-b64d-d12bf3212eae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:29.01711791 +0000 UTC m=+34.082709792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-prb2h" (UniqueName: "kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h") pod "network-check-target-z29nk" (UID: "bb6cf0be-ed34-4473-b64d-d12bf3212eae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:19:13.218407 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:13.218372 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:13.218559 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.218524 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 12:19:13.218613 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.218589 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret podName:a352b881-3036-4ed7-971a-22645878ebfd nodeName:}" failed. No retries permitted until 2026-08-05 12:19:29.218568132 +0000 UTC m=+34.284160033 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret") pod "global-pull-secret-syncer-c6j9w" (UID: "a352b881-3036-4ed7-971a-22645878ebfd") : object "kube-system"/"original-pull-secret" not registered Aug 05 12:19:13.437083 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:13.437006 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:13.437083 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:13.437040 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:13.437282 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:13.437006 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:13.437282 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.437143 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:13.437282 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.437222 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:13.437401 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:13.437308 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:15.437822 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:15.437751 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:15.438290 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:15.437859 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:15.438290 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:15.437864 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:15.438290 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:15.437990 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:15.438290 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:15.438027 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:15.438290 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:15.438107 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:17.437474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.436721 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:17.437474 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:17.436865 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:17.437474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.437235 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:17.437474 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:17.437326 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:17.437474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.437370 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:17.437474 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:17.437451 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:17.599941 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.599553 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" event={"ID":"6663f904-8a23-4be1-9d95-6c7b4585a35a","Type":"ContainerStarted","Data":"0fd7ca45c73d1692accf5f4fc4a350c2f30470b07a5c8dbe0d2b41ab63c01476"} Aug 05 12:19:17.601020 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.600989 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j7c8w" event={"ID":"00e242a0-2726-4da2-b46e-044a68d05c5d","Type":"ContainerStarted","Data":"026b39a0de680719bb7b9bb45b4e130a2793613b89c41b4b946fae358c9743b9"} Aug 05 12:19:17.602481 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.602444 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2fjj5" event={"ID":"61f27d7c-18bb-4986-98b1-0908ccfeec3e","Type":"ContainerStarted","Data":"2e818d01709ffef91662a1bc4cea1280ef00b8ebc4b7a33f758b5da518b6454f"} Aug 05 12:19:17.603918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.603892 2569 generic.go:356] "Generic (PLEG): container finished" podID="73786aea-c106-4636-9696-a5ec46bfeb24" containerID="5e0bac489e97c35daa3679bd724447b131e18aac4797f9db6bd94c657051836a" exitCode=0 Aug 05 12:19:17.604042 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.603931 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerDied","Data":"5e0bac489e97c35daa3679bd724447b131e18aac4797f9db6bd94c657051836a"} Aug 05 12:19:17.605902 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.605869 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"afc8a07baa39eb8960fa0d172f5b3671b86829fdfb3d7f91e07dbef63a40857a"} Aug 05 12:19:17.605997 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.605973 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"025e4a877cd917b004828ececb58948132430425f6f49ef4e791da04f8c7d84a"} Aug 05 12:19:17.607503 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.607481 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p286z" event={"ID":"b8d4ced4-b21e-4a12-8a35-269ad385fcc6","Type":"ContainerStarted","Data":"e6a839ae9af4f4ca43aae762efca44f6e77c0c222f95e962ee3450359da451c8"} Aug 05 12:19:17.609105 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.609027 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tp9z5" event={"ID":"d6a8082e-295e-4dc6-ae1e-35ce58a50e8f","Type":"ContainerStarted","Data":"eec24ca0697ac03acb7833a15f144ce7e1cfb753d242f76ed1d093ec36f3cf63"} Aug 05 12:19:17.611530 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.611510 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" event={"ID":"b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d","Type":"ContainerStarted","Data":"110414d89168bdb060afba5649454871169c280ff9fc7440ef5f2c56532f588b"} Aug 05 12:19:17.615691 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.615654 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j7c8w" podStartSLOduration=3.9687412 podStartE2EDuration="22.615641893s" totalImagesPullingTime="18.646900693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:17.615079719 +0000 UTC m=+22.680671617" watchObservedRunningTime="2026-08-05 12:19:17.615641893 +0000 UTC m=+22.681233788" Aug 05 12:19:17.630284 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.630240 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2fjj5" podStartSLOduration=3.000165448 podStartE2EDuration="21.630226119s" totalImagesPullingTime="18.630060671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:17.629833021 +0000 UTC m=+22.695424915" watchObservedRunningTime="2026-08-05 12:19:17.630226119 +0000 UTC m=+22.695818017" Aug 05 12:19:17.647907 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.647858 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gg2w9" podStartSLOduration=3.037541252 podStartE2EDuration="21.647840514s" totalImagesPullingTime="18.610299262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:17.646668761 +0000 UTC m=+22.712260659" watchObservedRunningTime="2026-08-05 12:19:17.647840514 +0000 UTC m=+22.713432413" Aug 05 12:19:17.690965 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.690895 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p286z" podStartSLOduration=3.057689023 podStartE2EDuration="21.690879835s" totalImagesPullingTime="18.633190812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:17.690802223 +0000 UTC m=+22.756394131" watchObservedRunningTime="2026-08-05 12:19:17.690879835 +0000 UTC m=+22.756471732" Aug 05 12:19:17.719388 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:17.719344 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tp9z5" podStartSLOduration=3.022045334 podStartE2EDuration="21.719329716s" totalImagesPullingTime="18.697284382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:17.719212215 +0000 UTC m=+22.784804113" watchObservedRunningTime="2026-08-05 12:19:17.719329716 +0000 UTC m=+22.784921614" Aug 05 12:19:18.205592 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.205528 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:19:18.206049 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.206035 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:19:18.620902 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.620868 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"540b33439617c41089f5652658d4f3ac3359fb3e617726b23671665483051a2c"} Aug 05 12:19:18.621690 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.620998 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"f5abc2bbc88a93265c37ea57f8cbc3a6c9e79ed7d6a171780514575d1bec1972"} Aug 05 12:19:18.621690 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.621038 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"a990593ca8d3a6061ea0abf592a6e4d42138c568bb60a0422e461c6130c22e04"} Aug 05 12:19:18.621690 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.621120 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"85ebf5c96079cd79e19579284ba330ea5ccd094f3d6d0a87a041db456fd37871"} Aug 05 12:19:18.622460 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.622430 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b27jc" event={"ID":"d3391f80-10ec-4a2c-abe5-a50d4cd2bc24","Type":"ContainerStarted","Data":"4723440e87dd75c3ce641827ad3d122d803f58a6f9542a97ef9020a808880e1a"} Aug 05 12:19:18.622728 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.622702 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:19:18.623299 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.623281 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2fjj5" Aug 05 12:19:18.652857 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.652797 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b27jc" podStartSLOduration=5.016081251 podStartE2EDuration="23.652778374s" totalImagesPullingTime="18.636697123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:18.65193936 +0000 UTC m=+23.717531258" watchObservedRunningTime="2026-08-05 12:19:18.652778374 +0000 UTC m=+23.718370273" Aug 05 12:19:18.810859 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:18.810822 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 05 12:19:19.339116 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:19.339022 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-05T12:19:18.810841145Z","UUID":"f3363f1a-041f-4505-9aaf-ff721caa28e6","Handler":null,"Name":"","Endpoint":""} Aug 05 12:19:19.340554 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:19.340533 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 05 12:19:19.340656 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:19.340576 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 05 12:19:19.437038 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:19.437009 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:19.437201 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:19.437009 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:19.437201 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:19.437138 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:19.437316 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:19.437236 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:19.437316 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:19.437011 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:19.437415 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:19.437325 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:19.627131 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:19.627046 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" event={"ID":"6663f904-8a23-4be1-9d95-6c7b4585a35a","Type":"ContainerStarted","Data":"999f17a0e5b3ff510ed8c49a004a710cfe28720a86ced17460f05c53356d1bb7"} Aug 05 12:19:20.632417 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:20.632379 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"d2c26303045c5f7c6fb52cef356e6aec0373e4fa7a7322d487a1d9150a34b544"} Aug 05 12:19:20.634137 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:20.634111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" event={"ID":"6663f904-8a23-4be1-9d95-6c7b4585a35a","Type":"ContainerStarted","Data":"e969d875f60ffa4061664c6273c0bb1df6a1896da4e5cf828f8bdb283e8be709"} Aug 05 12:19:20.652606 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:20.652566 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-www5s" podStartSLOduration=3.626610863 podStartE2EDuration="24.652553316s" totalImagesPullingTime="21.025942453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:20.652529589 +0000 UTC m=+25.718121489" watchObservedRunningTime="2026-08-05 12:19:20.652553316 +0000 UTC m=+25.718145196" Aug 05 12:19:21.436715 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:21.436677 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:21.436715 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:21.436697 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:21.437088 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:21.436678 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:21.437088 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:21.436814 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:21.437088 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:21.436898 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:21.437088 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:21.437023 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:22.641265 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:22.641090 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" event={"ID":"e4155f61-9591-437e-a077-b395af2f689e","Type":"ContainerStarted","Data":"09e372ae6a4abc65c68fced35cc047921b11cb832dca170e7f659cd47e887b05"} Aug 05 12:19:22.641870 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:22.641364 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:19:22.641870 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:22.641425 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:19:22.643006 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:22.642980 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerStarted","Data":"174c623def536672cdf25be7b6ee9cb4e27c37a587a731d868f64dc622e07268"} Aug 05 12:19:22.655425 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:22.655405 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:19:22.667157 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:22.667105 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" podStartSLOduration=7.989870771 podStartE2EDuration="26.667084715s" totalImagesPullingTime="18.677213944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:22.665387363 +0000 UTC m=+27.730979261" watchObservedRunningTime="2026-08-05 12:19:22.667084715 +0000 UTC m=+27.732676616" Aug 05 12:19:23.436158 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:23.436110 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:23.436158 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:23.436144 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:23.436366 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:23.436168 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:23.436366 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:23.436266 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:23.436447 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:23.436369 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:23.436481 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:23.436441 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:23.650107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:23.650031 2569 generic.go:356] "Generic (PLEG): container finished" podID="73786aea-c106-4636-9696-a5ec46bfeb24" containerID="174c623def536672cdf25be7b6ee9cb4e27c37a587a731d868f64dc622e07268" exitCode=0 Aug 05 12:19:23.650539 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:23.650106 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerDied","Data":"174c623def536672cdf25be7b6ee9cb4e27c37a587a731d868f64dc622e07268"} Aug 05 12:19:23.650653 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:23.650635 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:19:23.665070 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:23.665052 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:19:24.548009 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:24.547807 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c6j9w"] Aug 05 12:19:24.548179 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:24.548086 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:24.548246 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:24.548175 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:24.550943 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:24.550914 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vsh45"] Aug 05 12:19:24.551084 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:24.551047 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:24.551180 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:24.551160 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:24.551538 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:24.551516 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z29nk"] Aug 05 12:19:24.551634 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:24.551603 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:24.551708 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:24.551688 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:25.656431 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:25.656398 2569 generic.go:356] "Generic (PLEG): container finished" podID="73786aea-c106-4636-9696-a5ec46bfeb24" containerID="b1c5287759ae30b5ac0a71be722bd2fc9dfb17f13a8693a8f03861c8e761ff69" exitCode=0 Aug 05 12:19:25.656993 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:25.656482 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerDied","Data":"b1c5287759ae30b5ac0a71be722bd2fc9dfb17f13a8693a8f03861c8e761ff69"} Aug 05 12:19:26.436962 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:26.436913 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:26.437115 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:26.436918 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:26.437115 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:26.437071 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:26.437191 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:26.436921 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:26.437191 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:26.437139 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:26.437255 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:26.437231 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:27.663185 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:27.663156 2569 generic.go:356] "Generic (PLEG): container finished" podID="73786aea-c106-4636-9696-a5ec46bfeb24" containerID="0cf230fe059725a7ce9496d55ab42e39548b012789f9967a22cd55593ca3e9de" exitCode=0 Aug 05 12:19:27.663539 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:27.663195 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerDied","Data":"0cf230fe059725a7ce9496d55ab42e39548b012789f9967a22cd55593ca3e9de"} Aug 05 12:19:28.436034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:28.436008 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:28.436034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:28.436032 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:28.436219 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:28.436017 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:28.436219 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:28.436133 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsh45" podUID="bccda255-92ed-4f4c-b52f-f3e7c6a639bf" Aug 05 12:19:28.436219 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:28.436205 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z29nk" podUID="bb6cf0be-ed34-4473-b64d-d12bf3212eae" Aug 05 12:19:28.436372 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:28.436308 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c6j9w" podUID="a352b881-3036-4ed7-971a-22645878ebfd" Aug 05 12:19:29.040693 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:29.040473 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:29.041130 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:29.040745 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:29.041130 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:29.040628 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 12:19:29.041130 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:29.040804 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 12:19:29.041130 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:29.040825 2569 projected.go:196] Error preparing data for projected volume kube-api-access-prb2h for pod openshift-network-diagnostics/network-check-target-z29nk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:19:29.041130 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:29.040863 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:19:29.041130 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:29.040908 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h podName:bb6cf0be-ed34-4473-b64d-d12bf3212eae nodeName:}" failed. No retries permitted until 2026-08-05 12:20:01.040888109 +0000 UTC m=+66.106479993 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-prb2h" (UniqueName: "kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h") pod "network-check-target-z29nk" (UID: "bb6cf0be-ed34-4473-b64d-d12bf3212eae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 12:19:29.041130 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:29.040931 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs podName:bccda255-92ed-4f4c-b52f-f3e7c6a639bf nodeName:}" failed. No retries permitted until 2026-08-05 12:20:01.040919868 +0000 UTC m=+66.106511748 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs") pod "network-metrics-daemon-vsh45" (UID: "bccda255-92ed-4f4c-b52f-f3e7c6a639bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 12:19:29.241811 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:29.241776 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:29.241990 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:29.241897 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 12:19:29.241990 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:29.241961 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret podName:a352b881-3036-4ed7-971a-22645878ebfd nodeName:}" failed. No retries permitted until 2026-08-05 12:20:01.241935592 +0000 UTC m=+66.307527468 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret") pod "global-pull-secret-syncer-c6j9w" (UID: "a352b881-3036-4ed7-971a-22645878ebfd") : object "kube-system"/"original-pull-secret" not registered Aug 05 12:19:30.240329 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.240253 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-178.ec2.internal" event="NodeReady" Aug 05 12:19:30.240687 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.240382 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 05 12:19:30.279582 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.279296 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx"] Aug 05 12:19:30.281116 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.281094 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d86dcddf4-dgvbt"] Aug 05 12:19:30.281254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.281232 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx" Aug 05 12:19:30.283007 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.282990 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4"] Aug 05 12:19:30.283127 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.283110 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.284320 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.284298 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bv2z8\"" Aug 05 12:19:30.284487 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.284467 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.284740 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.284725 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.285270 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.285251 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.285359 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.285291 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 05 12:19:30.285359 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.285330 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vkpqd\"" Aug 05 12:19:30.285463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.285361 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 05 12:19:30.285772 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.285752 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 05 12:19:30.286992 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.286755 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Aug 05 12:19:30.286992 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.286779 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.288103 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.288082 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6b8rw\"" Aug 05 12:19:30.288205 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.288114 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Aug 05 12:19:30.288205 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.288158 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.291514 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.291493 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx"] Aug 05 12:19:30.291890 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.291865 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 05 12:19:30.298782 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.298762 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4"] Aug 05 12:19:30.299052 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.298910 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw"] Aug 05 12:19:30.301219 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.300831 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb"] Aug 05 12:19:30.301219 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.300996 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.305210 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.303898 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-86b82\"" Aug 05 12:19:30.305210 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.304467 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Aug 05 12:19:30.305210 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.304587 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Aug 05 12:19:30.305210 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.305122 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.305210 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.305174 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.306144 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.306120 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:30.308622 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.308603 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.308698 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.308677 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Aug 05 12:19:30.308809 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.308781 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.309140 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.309121 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-x65bl\"" Aug 05 12:19:30.312987 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.312967 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw"] Aug 05 12:19:30.313743 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.313714 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb"] Aug 05 12:19:30.317475 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.317455 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d86dcddf4-dgvbt"] Aug 05 12:19:30.319597 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.317883 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-xhr85"] Aug 05 12:19:30.321474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.321457 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z"] Aug 05 12:19:30.321595 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.321578 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:30.323292 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.323275 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wlqgr"] Aug 05 12:19:30.323426 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.323410 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.324252 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.323783 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Aug 05 12:19:30.324252 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.324224 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Aug 05 12:19:30.324376 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.324338 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ftpxl\"" Aug 05 12:19:30.325814 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.325794 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-z9wlp"] Aug 05 12:19:30.326131 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.326114 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.326336 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.326321 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Aug 05 12:19:30.326517 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.326469 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vjjhq\"" Aug 05 12:19:30.326730 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.326711 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Aug 05 12:19:30.326813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.326734 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.326813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.326749 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.327865 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.327848 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-7qgvq"] Aug 05 12:19:30.328075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.328049 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.328171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.328156 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 05 12:19:30.328708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.328639 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4c62j\"" Aug 05 12:19:30.328789 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.328716 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 05 12:19:30.329800 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.329717 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jtxvw"] Aug 05 12:19:30.331117 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.331099 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.332331 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.332126 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gkvp8\"" Aug 05 12:19:30.332535 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.332361 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Aug 05 12:19:30.332639 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.332539 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.332749 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.332714 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.334257 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.332788 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Aug 05 12:19:30.334257 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.333274 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7cc769bc88-2vr74"] Aug 05 12:19:30.337653 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.336622 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jtxvw" Aug 05 12:19:30.337653 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.337033 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.339443 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.338389 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Aug 05 12:19:30.339443 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.338561 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sqqg6\"" Aug 05 12:19:30.339443 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.338798 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Aug 05 12:19:30.339443 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.339097 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.339443 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.339296 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.340763 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.340735 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj"] Aug 05 12:19:30.341156 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.341042 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.342681 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.341507 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.343463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.343068 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 05 12:19:30.343463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.343159 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-swj9c\"" Aug 05 12:19:30.343463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.343373 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-xhr85"] Aug 05 12:19:30.343463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.343395 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z"] Aug 05 12:19:30.343463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.343408 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wlqgr"] Aug 05 12:19:30.343463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.343424 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-7qgvq"] Aug 05 12:19:30.346899 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.344108 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj"] Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.347553 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj" Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.347998 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jtxvw"] Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.348712 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.349077 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cc769bc88-2vr74"] Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.349192 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.350971 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.351197 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.351373 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.352192 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Aug 05 12:19:30.353620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.352786 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 05 12:19:30.354205 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.353858 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-z9wlp"] Aug 05 12:19:30.354205 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.354012 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 05 12:19:30.354454 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.354433 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Aug 05 12:19:30.354635 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.354620 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5l78w\"" Aug 05 12:19:30.356051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.356026 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Aug 05 12:19:30.356125 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.356052 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f64mq\"" Aug 05 12:19:30.436379 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.436353 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:19:30.436519 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.436461 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:19:30.436519 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.436487 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:19:30.438633 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.438591 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 05 12:19:30.438776 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.438663 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zm7jq\"" Aug 05 12:19:30.438776 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.438673 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 05 12:19:30.438776 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.438663 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rdzzt\"" Aug 05 12:19:30.450770 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.450752 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l94nq\" (UniqueName: \"kubernetes.io/projected/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-kube-api-access-l94nq\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.450853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.450789 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cf94-77ba-4851-8abb-2981237ae617-service-ca-bundle\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.450853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.450822 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eb4fa7e4-db72-4734-8a4b-2a55cc102395-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:30.450853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.450840 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-default-certificate\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.451031 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.450912 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpqzh\" (UniqueName: \"kubernetes.io/projected/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-kube-api-access-kpqzh\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.451081 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451005 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.451132 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451077 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2k2m\" (UniqueName: \"kubernetes.io/projected/b97663a6-9d83-437a-b7c7-13f5ab0445ae-kube-api-access-j2k2m\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.451132 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451105 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-tmp-dir\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.451236 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451154 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2739193d-8b60-4f26-9d22-f27389fcaab8-ca-trust-extracted\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.451236 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9lmk\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-kube-api-access-g9lmk\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.451236 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451222 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451277 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd67cf94-77ba-4851-8abb-2981237ae617-serving-cert\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451307 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451331 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451354 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-certificates\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451382 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqjq8\" (UniqueName: \"kubernetes.io/projected/cdd81e42-4928-48b5-9e19-296cb4774bef-kube-api-access-vqjq8\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451406 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-serving-cert\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451432 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451456 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-installation-pull-secrets\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451495 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b97663a6-9d83-437a-b7c7-13f5ab0445ae-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451548 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gw2h\" (UniqueName: \"kubernetes.io/projected/7a5ecb58-108a-4caf-a322-a8850ead53a5-kube-api-access-7gw2h\") pod \"volume-data-source-validator-5b8776669-rrnrx\" (UID: \"7a5ecb58-108a-4caf-a322-a8850ead53a5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451604 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5kjr\" (UniqueName: \"kubernetes.io/projected/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-kube-api-access-r5kjr\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451647 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451688 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-image-registry-private-configuration\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451706 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-bound-sa-token\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.451946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451722 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-config-volume\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451745 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-trusted-ca\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451809 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-serving-cert\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451846 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-config\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451883 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd67cf94-77ba-4851-8abb-2981237ae617-tmp\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451919 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js7rl\" (UniqueName: \"kubernetes.io/projected/dd67cf94-77ba-4851-8abb-2981237ae617-kube-api-access-js7rl\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.451978 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-stats-auth\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452039 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2gwp\" (UniqueName: \"kubernetes.io/projected/f71fe19e-7bc3-497c-a52e-f21eee789121-kube-api-access-n2gwp\") pod \"ingress-canary-jtxvw\" (UID: \"f71fe19e-7bc3-497c-a52e-f21eee789121\") " pod="openshift-ingress-canary/ingress-canary-jtxvw" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452066 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452100 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-trusted-ca\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452158 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbvqb\" (UniqueName: \"kubernetes.io/projected/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-kube-api-access-fbvqb\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452214 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v9gl\" (UniqueName: \"kubernetes.io/projected/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-kube-api-access-8v9gl\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452241 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92rnq\" (UniqueName: \"kubernetes.io/projected/552e4b27-d3d2-4ae2-9001-2be16d09b667-kube-api-access-92rnq\") pod \"network-check-source-65bdbbff45-dq9dj\" (UID: \"552e4b27-d3d2-4ae2-9001-2be16d09b667\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452267 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.452722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452298 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f71fe19e-7bc3-497c-a52e-f21eee789121-cert\") pod \"ingress-canary-jtxvw\" (UID: \"f71fe19e-7bc3-497c-a52e-f21eee789121\") " pod="openshift-ingress-canary/ingress-canary-jtxvw" Aug 05 12:19:30.453271 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452342 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-config\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.453271 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452371 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cf94-77ba-4851-8abb-2981237ae617-trusted-ca-bundle\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.453271 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.452418 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dd67cf94-77ba-4851-8abb-2981237ae617-snapshots\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.553596 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553564 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l94nq\" (UniqueName: \"kubernetes.io/projected/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-kube-api-access-l94nq\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.553761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553602 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cf94-77ba-4851-8abb-2981237ae617-service-ca-bundle\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.553761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553639 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eb4fa7e4-db72-4734-8a4b-2a55cc102395-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:30.553761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553662 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-default-certificate\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.553761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553685 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpqzh\" (UniqueName: \"kubernetes.io/projected/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-kube-api-access-kpqzh\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.553761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.553761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553736 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2k2m\" (UniqueName: \"kubernetes.io/projected/b97663a6-9d83-437a-b7c7-13f5ab0445ae-kube-api-access-j2k2m\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.553761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553759 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-tmp-dir\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553783 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2739193d-8b60-4f26-9d22-f27389fcaab8-ca-trust-extracted\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553806 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9lmk\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-kube-api-access-g9lmk\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553832 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553857 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd67cf94-77ba-4851-8abb-2981237ae617-serving-cert\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553886 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553910 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.553934 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-certificates\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.553996 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.554011 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d86dcddf4-dgvbt: secret "image-registry-tls" not found Aug 05 12:19:30.554143 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.554064 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls podName:2739193d-8b60-4f26-9d22-f27389fcaab8 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:31.054045869 +0000 UTC m=+36.119637759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls") pod "image-registry-5d86dcddf4-dgvbt" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8") : secret "image-registry-tls" not found Aug 05 12:19:30.554585 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554274 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cf94-77ba-4851-8abb-2981237ae617-service-ca-bundle\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.554585 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554485 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eb4fa7e4-db72-4734-8a4b-2a55cc102395-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:30.554585 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554553 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-certificates\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.554585 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.554567 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 05 12:19:30.554740 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.554616 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert podName:eb4fa7e4-db72-4734-8a4b-2a55cc102395 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:31.054603705 +0000 UTC m=+36.120195582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-xhr85" (UID: "eb4fa7e4-db72-4734-8a4b-2a55cc102395") : secret "networking-console-plugin-cert" not found Aug 05 12:19:30.554740 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554624 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.554740 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.554670 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 12:19:30.554740 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554682 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2739193d-8b60-4f26-9d22-f27389fcaab8-ca-trust-extracted\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.554740 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.554698 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls podName:5cc3ea90-8123-4429-a46e-ae3f0ab302c1 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:31.054687475 +0000 UTC m=+36.120279352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls") pod "dns-default-wlqgr" (UID: "5cc3ea90-8123-4429-a46e-ae3f0ab302c1") : secret "dns-default-metrics-tls" not found Aug 05 12:19:30.554937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554741 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqjq8\" (UniqueName: \"kubernetes.io/projected/cdd81e42-4928-48b5-9e19-296cb4774bef-kube-api-access-vqjq8\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:30.554937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554775 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-serving-cert\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.554937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554788 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-tmp-dir\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.554937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554806 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.554937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554833 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-installation-pull-secrets\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.554937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554860 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b97663a6-9d83-437a-b7c7-13f5ab0445ae-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.554937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554884 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gw2h\" (UniqueName: \"kubernetes.io/projected/7a5ecb58-108a-4caf-a322-a8850ead53a5-kube-api-access-7gw2h\") pod \"volume-data-source-validator-5b8776669-rrnrx\" (UID: \"7a5ecb58-108a-4caf-a322-a8850ead53a5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx" Aug 05 12:19:30.554937 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.554899 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:31.05488392 +0000 UTC m=+36.120475815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : configmap references non-existent config key: service-ca.crt Aug 05 12:19:30.555268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.554972 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5kjr\" (UniqueName: \"kubernetes.io/projected/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-kube-api-access-r5kjr\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.555268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555020 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.555268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555050 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-image-registry-private-configuration\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.555268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555077 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-bound-sa-token\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.555420 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.555403 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:30.555463 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.555458 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls podName:b97663a6-9d83-437a-b7c7-13f5ab0445ae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:31.05544328 +0000 UTC m=+36.121035157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-swmv4" (UID: "b97663a6-9d83-437a-b7c7-13f5ab0445ae") : secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:30.555519 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555508 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-config-volume\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.555566 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555535 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-trusted-ca\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.555609 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555572 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-serving-cert\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.555649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555607 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:30.555649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555630 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-config\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.555736 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd67cf94-77ba-4851-8abb-2981237ae617-tmp\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.555736 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555681 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js7rl\" (UniqueName: \"kubernetes.io/projected/dd67cf94-77ba-4851-8abb-2981237ae617-kube-api-access-js7rl\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.555736 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555703 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b97663a6-9d83-437a-b7c7-13f5ab0445ae-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.555736 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-stats-auth\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.555916 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555751 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2gwp\" (UniqueName: \"kubernetes.io/projected/f71fe19e-7bc3-497c-a52e-f21eee789121-kube-api-access-n2gwp\") pod \"ingress-canary-jtxvw\" (UID: \"f71fe19e-7bc3-497c-a52e-f21eee789121\") " pod="openshift-ingress-canary/ingress-canary-jtxvw" Aug 05 12:19:30.555916 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555777 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.555916 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555811 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-trusted-ca\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.555916 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.555818 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 12:19:30.555916 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbvqb\" (UniqueName: \"kubernetes.io/projected/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-kube-api-access-fbvqb\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.555916 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.555867 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls podName:cdd81e42-4928-48b5-9e19-296cb4774bef nodeName:}" failed. No retries permitted until 2026-08-05 12:19:31.055854102 +0000 UTC m=+36.121445979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-fxmhb" (UID: "cdd81e42-4928-48b5-9e19-296cb4774bef") : secret "samples-operator-tls" not found Aug 05 12:19:30.555916 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v9gl\" (UniqueName: \"kubernetes.io/projected/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-kube-api-access-8v9gl\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.556244 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555927 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92rnq\" (UniqueName: \"kubernetes.io/projected/552e4b27-d3d2-4ae2-9001-2be16d09b667-kube-api-access-92rnq\") pod \"network-check-source-65bdbbff45-dq9dj\" (UID: \"552e4b27-d3d2-4ae2-9001-2be16d09b667\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj" Aug 05 12:19:30.556244 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.555995 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.556244 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.556021 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f71fe19e-7bc3-497c-a52e-f21eee789121-cert\") pod \"ingress-canary-jtxvw\" (UID: \"f71fe19e-7bc3-497c-a52e-f21eee789121\") " pod="openshift-ingress-canary/ingress-canary-jtxvw" Aug 05 12:19:30.556244 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.556049 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-config\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.556244 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.556091 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cf94-77ba-4851-8abb-2981237ae617-trusted-ca-bundle\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.556244 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.556124 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dd67cf94-77ba-4851-8abb-2981237ae617-snapshots\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.556519 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.556349 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd67cf94-77ba-4851-8abb-2981237ae617-tmp\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.556641 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.556615 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dd67cf94-77ba-4851-8abb-2981237ae617-snapshots\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.556641 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.556631 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-config-volume\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.556878 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.556854 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-config\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.557005 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.556965 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 05 12:19:30.557070 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:30.557017 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:31.057002803 +0000 UTC m=+36.122594681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : secret "router-metrics-certs-default" not found Aug 05 12:19:30.557804 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.557775 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-config\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.557904 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.557875 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-trusted-ca\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.558940 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.558123 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-trusted-ca\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.558940 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.558893 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cf94-77ba-4851-8abb-2981237ae617-trusted-ca-bundle\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.559169 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.559148 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dd67cf94-77ba-4851-8abb-2981237ae617-serving-cert\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.559413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.559388 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-stats-auth\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.560216 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.560196 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-default-certificate\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.560344 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.560203 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-installation-pull-secrets\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.560344 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.560254 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-image-registry-private-configuration\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.560344 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.560265 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-serving-cert\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.560527 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.560503 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-serving-cert\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.560897 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.560852 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.562234 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.562188 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f71fe19e-7bc3-497c-a52e-f21eee789121-cert\") pod \"ingress-canary-jtxvw\" (UID: \"f71fe19e-7bc3-497c-a52e-f21eee789121\") " pod="openshift-ingress-canary/ingress-canary-jtxvw" Aug 05 12:19:30.564056 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.564037 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l94nq\" (UniqueName: \"kubernetes.io/projected/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-kube-api-access-l94nq\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:30.567044 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.566648 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqjq8\" (UniqueName: \"kubernetes.io/projected/cdd81e42-4928-48b5-9e19-296cb4774bef-kube-api-access-vqjq8\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:30.567044 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.567007 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-bound-sa-token\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.567364 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.567342 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2gwp\" (UniqueName: \"kubernetes.io/projected/f71fe19e-7bc3-497c-a52e-f21eee789121-kube-api-access-n2gwp\") pod \"ingress-canary-jtxvw\" (UID: \"f71fe19e-7bc3-497c-a52e-f21eee789121\") " pod="openshift-ingress-canary/ingress-canary-jtxvw" Aug 05 12:19:30.568092 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.568069 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2k2m\" (UniqueName: \"kubernetes.io/projected/b97663a6-9d83-437a-b7c7-13f5ab0445ae-kube-api-access-j2k2m\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:30.569488 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.568306 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92rnq\" (UniqueName: \"kubernetes.io/projected/552e4b27-d3d2-4ae2-9001-2be16d09b667-kube-api-access-92rnq\") pod \"network-check-source-65bdbbff45-dq9dj\" (UID: \"552e4b27-d3d2-4ae2-9001-2be16d09b667\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj" Aug 05 12:19:30.569488 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.569217 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gw2h\" (UniqueName: \"kubernetes.io/projected/7a5ecb58-108a-4caf-a322-a8850ead53a5-kube-api-access-7gw2h\") pod \"volume-data-source-validator-5b8776669-rrnrx\" (UID: \"7a5ecb58-108a-4caf-a322-a8850ead53a5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx" Aug 05 12:19:30.569488 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.569348 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpqzh\" (UniqueName: \"kubernetes.io/projected/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-kube-api-access-kpqzh\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:30.569687 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.569561 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9lmk\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-kube-api-access-g9lmk\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:30.574417 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.569886 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5kjr\" (UniqueName: \"kubernetes.io/projected/bbc43ec4-08bb-4c69-b4e2-94b3e60eb797-kube-api-access-r5kjr\") pod \"console-operator-5ccd6b445f-z9wlp\" (UID: \"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797\") " pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.574417 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.570060 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbvqb\" (UniqueName: \"kubernetes.io/projected/4352fed2-2dde-4ff6-9cb7-998b3fded2e7-kube-api-access-fbvqb\") pod \"service-ca-operator-784f97bcf6-lxj6z\" (UID: \"4352fed2-2dde-4ff6-9cb7-998b3fded2e7\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.574417 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.570137 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v9gl\" (UniqueName: \"kubernetes.io/projected/1e3ef6c1-6fc9-4467-ae88-65c7b7b24436-kube-api-access-8v9gl\") pod \"kube-storage-version-migrator-operator-7878f4b88c-s8grw\" (UID: \"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.576127 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.576047 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js7rl\" (UniqueName: \"kubernetes.io/projected/dd67cf94-77ba-4851-8abb-2981237ae617-kube-api-access-js7rl\") pod \"insights-operator-77696685b-7qgvq\" (UID: \"dd67cf94-77ba-4851-8abb-2981237ae617\") " pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.594189 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.594172 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx" Aug 05 12:19:30.623041 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.623015 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" Aug 05 12:19:30.657050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.657021 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" Aug 05 12:19:30.673728 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.673576 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:30.679589 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.679570 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-7qgvq" Aug 05 12:19:30.686275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.686261 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jtxvw" Aug 05 12:19:30.699026 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:30.699007 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj" Aug 05 12:19:31.060213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:31.060173 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:31.060396 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:31.060227 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:31.060396 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:31.060261 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:31.060396 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:31.060289 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:31.060396 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060295 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 12:19:31.060396 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060320 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d86dcddf4-dgvbt: secret "image-registry-tls" not found Aug 05 12:19:31.060396 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:31.060339 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:31.060396 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060380 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060380 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls podName:2739193d-8b60-4f26-9d22-f27389fcaab8 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:32.060360712 +0000 UTC m=+37.125952610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls") pod "image-registry-5d86dcddf4-dgvbt" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8") : secret "image-registry-tls" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060425 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060431 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060441 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert podName:eb4fa7e4-db72-4734-8a4b-2a55cc102395 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:32.060429788 +0000 UTC m=+37.126021668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-xhr85" (UID: "eb4fa7e4-db72-4734-8a4b-2a55cc102395") : secret "networking-console-plugin-cert" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:31.060491 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060504 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:32.060493906 +0000 UTC m=+37.126085800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : configmap references non-existent config key: service-ca.crt Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060528 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls podName:b97663a6-9d83-437a-b7c7-13f5ab0445ae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:32.060519748 +0000 UTC m=+37.126111635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-swmv4" (UID: "b97663a6-9d83-437a-b7c7-13f5ab0445ae") : secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060544 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls podName:5cc3ea90-8123-4429-a46e-ae3f0ab302c1 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:32.060536683 +0000 UTC m=+37.126128559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls") pod "dns-default-wlqgr" (UID: "5cc3ea90-8123-4429-a46e-ae3f0ab302c1") : secret "dns-default-metrics-tls" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060566 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:31.060578 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060627 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls podName:cdd81e42-4928-48b5-9e19-296cb4774bef nodeName:}" failed. No retries permitted until 2026-08-05 12:19:32.060611527 +0000 UTC m=+37.126203403 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-fxmhb" (UID: "cdd81e42-4928-48b5-9e19-296cb4774bef") : secret "samples-operator-tls" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060657 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 05 12:19:31.060696 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:31.060696 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:32.060687066 +0000 UTC m=+37.126278942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : secret "router-metrics-certs-default" not found Aug 05 12:19:32.056430 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.055711 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx"] Aug 05 12:19:32.064743 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:19:32.061937 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a5ecb58_108a_4caf_a322_a8850ead53a5.slice/crio-01b09c4a3314f7b148bfa12ba3e4e6528e5721ba532602005c2dcb3fca7e8cc5 WatchSource:0}: Error finding container 01b09c4a3314f7b148bfa12ba3e4e6528e5721ba532602005c2dcb3fca7e8cc5: Status 404 returned error can't find the container with id 01b09c4a3314f7b148bfa12ba3e4e6528e5721ba532602005c2dcb3fca7e8cc5 Aug 05 12:19:32.077296 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.077267 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:32.077409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.077320 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:32.077469 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.077409 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:32.077469 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.077446 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:32.077567 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.077478 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:32.077567 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.077512 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:32.077653 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.077574 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:32.077735 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.077719 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.077785 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls podName:b97663a6-9d83-437a-b7c7-13f5ab0445ae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:34.077766145 +0000 UTC m=+39.143358028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-swmv4" (UID: "b97663a6-9d83-437a-b7c7-13f5ab0445ae") : secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078043 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078113 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert podName:eb4fa7e4-db72-4734-8a4b-2a55cc102395 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:34.078092875 +0000 UTC m=+39.143684961 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-xhr85" (UID: "eb4fa7e4-db72-4734-8a4b-2a55cc102395") : secret "networking-console-plugin-cert" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078194 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078240 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls podName:cdd81e42-4928-48b5-9e19-296cb4774bef nodeName:}" failed. No retries permitted until 2026-08-05 12:19:34.078226173 +0000 UTC m=+39.143818050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-fxmhb" (UID: "cdd81e42-4928-48b5-9e19-296cb4774bef") : secret "samples-operator-tls" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078302 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078338 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:34.078326944 +0000 UTC m=+39.143918823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : secret "router-metrics-certs-default" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078408 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078420 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d86dcddf4-dgvbt: secret "image-registry-tls" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078458 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls podName:2739193d-8b60-4f26-9d22-f27389fcaab8 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:34.078443468 +0000 UTC m=+39.144035345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls") pod "image-registry-5d86dcddf4-dgvbt" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8") : secret "image-registry-tls" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078538 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:34.078526461 +0000 UTC m=+39.144118339 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : configmap references non-existent config key: service-ca.crt Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078599 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 12:19:32.078906 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:32.078632 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls podName:5cc3ea90-8123-4429-a46e-ae3f0ab302c1 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:34.078620064 +0000 UTC m=+39.144211947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls") pod "dns-default-wlqgr" (UID: "5cc3ea90-8123-4429-a46e-ae3f0ab302c1") : secret "dns-default-metrics-tls" not found Aug 05 12:19:32.081481 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.081442 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z"] Aug 05 12:19:32.104198 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.104168 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-z9wlp"] Aug 05 12:19:32.111286 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.111263 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-7qgvq"] Aug 05 12:19:32.114290 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.114269 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw"] Aug 05 12:19:32.119806 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.119764 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jtxvw"] Aug 05 12:19:32.121189 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.121106 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj"] Aug 05 12:19:32.123006 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:19:32.122982 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf71fe19e_7bc3_497c_a52e_f21eee789121.slice/crio-fda8cd9ece06e178d7afc20392991e6b2bc1b6850e3ac803bc1258706987a9df WatchSource:0}: Error finding container fda8cd9ece06e178d7afc20392991e6b2bc1b6850e3ac803bc1258706987a9df: Status 404 returned error can't find the container with id fda8cd9ece06e178d7afc20392991e6b2bc1b6850e3ac803bc1258706987a9df Aug 05 12:19:32.124069 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:19:32.124047 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd67cf94_77ba_4851_8abb_2981237ae617.slice/crio-3ca448d67e3eb949198faf3328caa3205f0aa89d241877a1764cc06d761b1741 WatchSource:0}: Error finding container 3ca448d67e3eb949198faf3328caa3205f0aa89d241877a1764cc06d761b1741: Status 404 returned error can't find the container with id 3ca448d67e3eb949198faf3328caa3205f0aa89d241877a1764cc06d761b1741 Aug 05 12:19:32.125142 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:19:32.124906 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod552e4b27_d3d2_4ae2_9001_2be16d09b667.slice/crio-ec3c03457207508a92751d73e5a44dc2193a56cf4a5d468aceed7c39875ea07f WatchSource:0}: Error finding container ec3c03457207508a92751d73e5a44dc2193a56cf4a5d468aceed7c39875ea07f: Status 404 returned error can't find the container with id ec3c03457207508a92751d73e5a44dc2193a56cf4a5d468aceed7c39875ea07f Aug 05 12:19:32.126669 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:19:32.126583 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc43ec4_08bb_4c69_b4e2_94b3e60eb797.slice/crio-917c0f763fbe44ddbf658c9f02c7b0f8d519375c5e596bfa1701b8f953d10dfe WatchSource:0}: Error finding container 917c0f763fbe44ddbf658c9f02c7b0f8d519375c5e596bfa1701b8f953d10dfe: Status 404 returned error can't find the container with id 917c0f763fbe44ddbf658c9f02c7b0f8d519375c5e596bfa1701b8f953d10dfe Aug 05 12:19:32.127349 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:19:32.127145 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4352fed2_2dde_4ff6_9cb7_998b3fded2e7.slice/crio-5c7d8fff4e75fb2efc6917a22dc8df8170702cc0f7490b84dc71e6ac33fd2d85 WatchSource:0}: Error finding container 5c7d8fff4e75fb2efc6917a22dc8df8170702cc0f7490b84dc71e6ac33fd2d85: Status 404 returned error can't find the container with id 5c7d8fff4e75fb2efc6917a22dc8df8170702cc0f7490b84dc71e6ac33fd2d85 Aug 05 12:19:32.129895 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:19:32.129115 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-ba2e59ca7dcef47a6168feec99123a785648ee2f538b7341730566d95dec7262 WatchSource:0}: Error finding container ba2e59ca7dcef47a6168feec99123a785648ee2f538b7341730566d95dec7262: Status 404 returned error can't find the container with id ba2e59ca7dcef47a6168feec99123a785648ee2f538b7341730566d95dec7262 Aug 05 12:19:32.682350 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.682305 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" event={"ID":"4352fed2-2dde-4ff6-9cb7-998b3fded2e7","Type":"ContainerStarted","Data":"5c7d8fff4e75fb2efc6917a22dc8df8170702cc0f7490b84dc71e6ac33fd2d85"} Aug 05 12:19:32.684589 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.684501 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" event={"ID":"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436","Type":"ContainerStarted","Data":"ba2e59ca7dcef47a6168feec99123a785648ee2f538b7341730566d95dec7262"} Aug 05 12:19:32.691613 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.690698 2569 generic.go:356] "Generic (PLEG): container finished" podID="73786aea-c106-4636-9696-a5ec46bfeb24" containerID="9bd9ca72982c2eb7325f693df3049e785a546e5940f2ee8e192f59f457d7194a" exitCode=0 Aug 05 12:19:32.691613 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.690806 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerDied","Data":"9bd9ca72982c2eb7325f693df3049e785a546e5940f2ee8e192f59f457d7194a"} Aug 05 12:19:32.693417 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.693353 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" event={"ID":"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797","Type":"ContainerStarted","Data":"917c0f763fbe44ddbf658c9f02c7b0f8d519375c5e596bfa1701b8f953d10dfe"} Aug 05 12:19:32.695708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.695622 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jtxvw" event={"ID":"f71fe19e-7bc3-497c-a52e-f21eee789121","Type":"ContainerStarted","Data":"fda8cd9ece06e178d7afc20392991e6b2bc1b6850e3ac803bc1258706987a9df"} Aug 05 12:19:32.714126 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.706783 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-7qgvq" event={"ID":"dd67cf94-77ba-4851-8abb-2981237ae617","Type":"ContainerStarted","Data":"3ca448d67e3eb949198faf3328caa3205f0aa89d241877a1764cc06d761b1741"} Aug 05 12:19:32.714126 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.708841 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx" event={"ID":"7a5ecb58-108a-4caf-a322-a8850ead53a5","Type":"ContainerStarted","Data":"01b09c4a3314f7b148bfa12ba3e4e6528e5721ba532602005c2dcb3fca7e8cc5"} Aug 05 12:19:32.714126 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:32.711569 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj" event={"ID":"552e4b27-d3d2-4ae2-9001-2be16d09b667","Type":"ContainerStarted","Data":"ec3c03457207508a92751d73e5a44dc2193a56cf4a5d468aceed7c39875ea07f"} Aug 05 12:19:33.723002 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:33.722965 2569 generic.go:356] "Generic (PLEG): container finished" podID="73786aea-c106-4636-9696-a5ec46bfeb24" containerID="470793e64fa5c70661fd2361d52dcdfdaf64c54c02be5a279bafab4340e091e5" exitCode=0 Aug 05 12:19:33.723433 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:33.723038 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerDied","Data":"470793e64fa5c70661fd2361d52dcdfdaf64c54c02be5a279bafab4340e091e5"} Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.103294 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.103634 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.103745 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.103780 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.103812 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.103845 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.103902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.103512 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104099 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls podName:cdd81e42-4928-48b5-9e19-296cb4774bef nodeName:}" failed. No retries permitted until 2026-08-05 12:19:38.10408217 +0000 UTC m=+43.169674050 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-fxmhb" (UID: "cdd81e42-4928-48b5-9e19-296cb4774bef") : secret "samples-operator-tls" not found Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104165 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104197 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:38.104186284 +0000 UTC m=+43.169778166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : secret "router-metrics-certs-default" not found Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104266 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104277 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d86dcddf4-dgvbt: secret "image-registry-tls" not found Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104309 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls podName:2739193d-8b60-4f26-9d22-f27389fcaab8 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:38.104296023 +0000 UTC m=+43.169887916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls") pod "image-registry-5d86dcddf4-dgvbt" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8") : secret "image-registry-tls" not found Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104368 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 05 12:19:34.105017 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104401 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert podName:eb4fa7e4-db72-4734-8a4b-2a55cc102395 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:38.104391572 +0000 UTC m=+43.169983461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-xhr85" (UID: "eb4fa7e4-db72-4734-8a4b-2a55cc102395") : secret "networking-console-plugin-cert" not found Aug 05 12:19:34.107392 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104464 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 12:19:34.107392 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104495 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls podName:5cc3ea90-8123-4429-a46e-ae3f0ab302c1 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:38.104484599 +0000 UTC m=+43.170076480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls") pod "dns-default-wlqgr" (UID: "5cc3ea90-8123-4429-a46e-ae3f0ab302c1") : secret "dns-default-metrics-tls" not found Aug 05 12:19:34.107392 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104564 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:38.104553262 +0000 UTC m=+43.170145141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : configmap references non-existent config key: service-ca.crt Aug 05 12:19:34.107392 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104040 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:34.107392 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:34.104600 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls podName:b97663a6-9d83-437a-b7c7-13f5ab0445ae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:38.104591514 +0000 UTC m=+43.170183393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-swmv4" (UID: "b97663a6-9d83-437a-b7c7-13f5ab0445ae") : secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:34.528036 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.528003 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:34.633970 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.633880 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:34.733025 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.732935 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" event={"ID":"73786aea-c106-4636-9696-a5ec46bfeb24","Type":"ContainerStarted","Data":"454d6b9ff9d504791f28c2f0aad887f7bd64e8cdf996d4893470ba319b8e9a3c"} Aug 05 12:19:34.735879 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.735411 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx" event={"ID":"7a5ecb58-108a-4caf-a322-a8850ead53a5","Type":"ContainerStarted","Data":"d6a36eae60d4a338ea399ccc950e8c1966e8a4590f2a581c261cdb6a3e4066aa"} Aug 05 12:19:34.759680 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.759073 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w4ll9" podStartSLOduration=9.885059924 podStartE2EDuration="39.759050882s" totalImagesPullingTime="29.873990958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:34.757045617 +0000 UTC m=+39.822637516" watchObservedRunningTime="2026-08-05 12:19:34.759050882 +0000 UTC m=+39.824642781" Aug 05 12:19:34.774664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.774609 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-rrnrx" podStartSLOduration=35.936700159 podStartE2EDuration="37.774591215s" totalImagesPullingTime="1.837891056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:34.772823684 +0000 UTC m=+39.838415601" watchObservedRunningTime="2026-08-05 12:19:34.774591215 +0000 UTC m=+39.840183114" Aug 05 12:19:34.878864 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.878781 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:34.980596 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:34.980516 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:35.042421 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:35.042389 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:35.517102 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:35.517066 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:35.618453 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:35.618424 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:36.280782 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:36.280745 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:36.565469 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:36.565401 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:36.637132 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:36.637099 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:38.140283 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:38.140251 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140413 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:38.140417 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140479 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140495 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d86dcddf4-dgvbt: secret "image-registry-tls" not found Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140484 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:46.140466834 +0000 UTC m=+51.206058730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : secret "router-metrics-certs-default" not found Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140538 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls podName:2739193d-8b60-4f26-9d22-f27389fcaab8 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:46.140527719 +0000 UTC m=+51.206119607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls") pod "image-registry-5d86dcddf4-dgvbt" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8") : secret "image-registry-tls" not found Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:38.140578 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:38.140611 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:38.140657 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:38.140642 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:38.140695 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140719 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:38.140733 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140752 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140775 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls podName:5cc3ea90-8123-4429-a46e-ae3f0ab302c1 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:46.140758292 +0000 UTC m=+51.206350189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls") pod "dns-default-wlqgr" (UID: "5cc3ea90-8123-4429-a46e-ae3f0ab302c1") : secret "dns-default-metrics-tls" not found Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140830 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:19:46.140812493 +0000 UTC m=+51.206404379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : configmap references non-existent config key: service-ca.crt Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140832 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140857 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert podName:eb4fa7e4-db72-4734-8a4b-2a55cc102395 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:46.140847217 +0000 UTC m=+51.206439100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-xhr85" (UID: "eb4fa7e4-db72-4734-8a4b-2a55cc102395") : secret "networking-console-plugin-cert" not found Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140847 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140921 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls podName:cdd81e42-4928-48b5-9e19-296cb4774bef nodeName:}" failed. No retries permitted until 2026-08-05 12:19:46.140908359 +0000 UTC m=+51.206500238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-fxmhb" (UID: "cdd81e42-4928-48b5-9e19-296cb4774bef") : secret "samples-operator-tls" not found Aug 05 12:19:38.141075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:38.140988 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls podName:b97663a6-9d83-437a-b7c7-13f5ab0445ae nodeName:}" failed. No retries permitted until 2026-08-05 12:19:46.140977299 +0000 UTC m=+51.206569189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-swmv4" (UID: "b97663a6-9d83-437a-b7c7-13f5ab0445ae") : secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:39.750766 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.750324 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" event={"ID":"4352fed2-2dde-4ff6-9cb7-998b3fded2e7","Type":"ContainerStarted","Data":"2b40168a1462703c041362f6e363328476f823849927adbd6196f3fd82f57022"} Aug 05 12:19:39.752332 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.752302 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" event={"ID":"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797","Type":"ContainerStarted","Data":"85c03c63f3f964ad35d808c9ff1acf898c64eaaf2e0bc64ead16ce50d00af425"} Aug 05 12:19:39.754112 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.754093 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:39.754571 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.754261 2569 patch_prober.go:28] interesting pod/console-operator-5ccd6b445f-z9wlp container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.18:8443/readyz\": dial tcp 10.132.0.18:8443: connect: connection refused" start-of-body= Aug 05 12:19:39.754571 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.754310 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" podUID="bbc43ec4-08bb-4c69-b4e2-94b3e60eb797" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.18:8443/readyz\": dial tcp 10.132.0.18:8443: connect: connection refused" Aug 05 12:19:39.756087 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.756061 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-7qgvq" event={"ID":"dd67cf94-77ba-4851-8abb-2981237ae617","Type":"ContainerStarted","Data":"150ccd40cb64660e5f9a7d6ae53ac82e11959e15880298681e82d7c12bfcf70c"} Aug 05 12:19:39.767409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.767368 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" podStartSLOduration=35.301343627 podStartE2EDuration="42.76735498s" totalImagesPullingTime="7.466011353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:39.766630434 +0000 UTC m=+44.832222333" watchObservedRunningTime="2026-08-05 12:19:39.76735498 +0000 UTC m=+44.832946877" Aug 05 12:19:39.784714 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.784652 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-7qgvq" podStartSLOduration=35.315687126 podStartE2EDuration="42.784632176s" totalImagesPullingTime="7.46894505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:39.784386564 +0000 UTC m=+44.849978460" watchObservedRunningTime="2026-08-05 12:19:39.784632176 +0000 UTC m=+44.850224079" Aug 05 12:19:39.805688 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:39.805525 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" podStartSLOduration=35.338349004 podStartE2EDuration="42.805510737s" totalImagesPullingTime="7.467161733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:39.805240627 +0000 UTC m=+44.870832527" watchObservedRunningTime="2026-08-05 12:19:39.805510737 +0000 UTC m=+44.871102639" Aug 05 12:19:40.674911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.674875 2569 patch_prober.go:28] interesting pod/console-operator-5ccd6b445f-z9wlp container/console-operator namespace/openshift-console-operator: Liveness probe status=failure output="Get \"https://10.132.0.18:8443/healthz\": dial tcp 10.132.0.18:8443: connect: connection refused" start-of-body= Aug 05 12:19:40.675062 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.674935 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" podUID="bbc43ec4-08bb-4c69-b4e2-94b3e60eb797" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.18:8443/healthz\": dial tcp 10.132.0.18:8443: connect: connection refused" Aug 05 12:19:40.762252 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.762216 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" event={"ID":"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436","Type":"ContainerStarted","Data":"db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a"} Aug 05 12:19:40.765214 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.765188 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/0.log" Aug 05 12:19:40.765352 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.765231 2569 generic.go:356] "Generic (PLEG): container finished" podID="bbc43ec4-08bb-4c69-b4e2-94b3e60eb797" containerID="85c03c63f3f964ad35d808c9ff1acf898c64eaaf2e0bc64ead16ce50d00af425" exitCode=255 Aug 05 12:19:40.765352 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.765304 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" event={"ID":"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797","Type":"ContainerDied","Data":"85c03c63f3f964ad35d808c9ff1acf898c64eaaf2e0bc64ead16ce50d00af425"} Aug 05 12:19:40.765553 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.765534 2569 scope.go:117] "RemoveContainer" containerID="85c03c63f3f964ad35d808c9ff1acf898c64eaaf2e0bc64ead16ce50d00af425" Aug 05 12:19:40.767012 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.766982 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jtxvw" event={"ID":"f71fe19e-7bc3-497c-a52e-f21eee789121","Type":"ContainerStarted","Data":"104dea0a2baa6d617957699e23767eb1b871ad8646c4aef65894a1c6a72750ae"} Aug 05 12:19:40.768741 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.768719 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj" event={"ID":"552e4b27-d3d2-4ae2-9001-2be16d09b667","Type":"ContainerStarted","Data":"af59ac85a1ed90e2a6af11ee98cf5995ddc5503f90cd2e9616c8b1236f6409ce"} Aug 05 12:19:40.774946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.774925 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:40.780665 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.780616 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" podStartSLOduration=36.316201455 podStartE2EDuration="43.780599826s" totalImagesPullingTime="7.464398371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:40.779783882 +0000 UTC m=+45.845375782" watchObservedRunningTime="2026-08-05 12:19:40.780599826 +0000 UTC m=+45.846191724" Aug 05 12:19:40.797181 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.795704 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jtxvw" podStartSLOduration=3.325787159 podStartE2EDuration="10.795686686s" totalImagesPullingTime="7.469899527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:40.795349271 +0000 UTC m=+45.860941170" watchObservedRunningTime="2026-08-05 12:19:40.795686686 +0000 UTC m=+45.861278585" Aug 05 12:19:40.813723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.813597 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-dq9dj" podStartSLOduration=36.334563984 podStartE2EDuration="43.813578885s" totalImagesPullingTime="7.479014901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:19:40.812829471 +0000 UTC m=+45.878421374" watchObservedRunningTime="2026-08-05 12:19:40.813578885 +0000 UTC m=+45.879170784" Aug 05 12:19:40.985786 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:40.985754 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.060342 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.060314 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.130581 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.130552 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.235769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.235691 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.294225 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.294196 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.477808 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.477765 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.579534 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.579497 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.737095 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.737064 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.746410 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.746385 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-shpvc"] Aug 05 12:19:41.767824 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.767802 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-shpvc"] Aug 05 12:19:41.768200 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.767907 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.773142 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.773116 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 05 12:19:41.773285 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.773262 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 05 12:19:41.774347 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.774175 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f4rtr\"" Aug 05 12:19:41.774447 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.774427 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:19:41.774989 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.774945 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/0.log" Aug 05 12:19:41.775074 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.775010 2569 generic.go:356] "Generic (PLEG): container finished" podID="bbc43ec4-08bb-4c69-b4e2-94b3e60eb797" containerID="2f7f68b5445cbd926aff3423cd61c308b57634e6df6877d1817773916198c5ee" exitCode=255 Aug 05 12:19:41.775166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.775145 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" event={"ID":"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797","Type":"ContainerDied","Data":"2f7f68b5445cbd926aff3423cd61c308b57634e6df6877d1817773916198c5ee"} Aug 05 12:19:41.775227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.775204 2569 scope.go:117] "RemoveContainer" containerID="85c03c63f3f964ad35d808c9ff1acf898c64eaaf2e0bc64ead16ce50d00af425" Aug 05 12:19:41.776848 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.776835 2569 scope.go:117] "RemoveContainer" containerID="2f7f68b5445cbd926aff3423cd61c308b57634e6df6877d1817773916198c5ee" Aug 05 12:19:41.777052 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:41.777029 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-z9wlp_openshift-console-operator(bbc43ec4-08bb-4c69-b4e2-94b3e60eb797)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" podUID="bbc43ec4-08bb-4c69-b4e2-94b3e60eb797" Aug 05 12:19:41.875533 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.874985 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab686670-c502-4b04-aaad-6cb41a462119-crio-socket\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.875710 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.875686 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqkdn\" (UniqueName: \"kubernetes.io/projected/ab686670-c502-4b04-aaad-6cb41a462119-kube-api-access-lqkdn\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.875788 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.875756 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.875839 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.875827 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab686670-c502-4b04-aaad-6cb41a462119-data-volume\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.875890 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.875860 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.890068 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.890040 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:41.976342 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.976316 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab686670-c502-4b04-aaad-6cb41a462119-crio-socket\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.976435 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.976366 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqkdn\" (UniqueName: \"kubernetes.io/projected/ab686670-c502-4b04-aaad-6cb41a462119-kube-api-access-lqkdn\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.976435 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.976384 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.976435 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.976411 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab686670-c502-4b04-aaad-6cb41a462119-data-volume\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.976435 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.976426 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.976581 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.976427 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab686670-c502-4b04-aaad-6cb41a462119-crio-socket\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.976581 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:41.976479 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 12:19:41.976581 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:41.976528 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:42.476511091 +0000 UTC m=+47.542102967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : configmap "kube-rbac-proxy" not found Aug 05 12:19:41.976581 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:41.976549 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 12:19:41.976716 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:41.976595 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:42.47657934 +0000 UTC m=+47.542171236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : secret "insights-runtime-extractor-tls" not found Aug 05 12:19:41.976756 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.976734 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab686670-c502-4b04-aaad-6cb41a462119-data-volume\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:41.989780 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:41.989761 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqkdn\" (UniqueName: \"kubernetes.io/projected/ab686670-c502-4b04-aaad-6cb41a462119-kube-api-access-lqkdn\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:42.480430 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:42.480396 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:42.480578 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:42.480444 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:42.480578 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:42.480544 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 12:19:42.480651 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:42.480619 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:43.480603551 +0000 UTC m=+48.546195427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : secret "insights-runtime-extractor-tls" not found Aug 05 12:19:42.480651 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:42.480546 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 12:19:42.480719 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:42.480680 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:43.480666604 +0000 UTC m=+48.546258497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : configmap "kube-rbac-proxy" not found Aug 05 12:19:42.779034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:42.779015 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:19:42.779354 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:42.779333 2569 scope.go:117] "RemoveContainer" containerID="2f7f68b5445cbd926aff3423cd61c308b57634e6df6877d1817773916198c5ee" Aug 05 12:19:42.779520 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:42.779499 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-z9wlp_openshift-console-operator(bbc43ec4-08bb-4c69-b4e2-94b3e60eb797)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" podUID="bbc43ec4-08bb-4c69-b4e2-94b3e60eb797" Aug 05 12:19:43.111551 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:43.111480 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:43.168826 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:43.168791 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:43.488575 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:43.488501 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:43.488575 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:43.488545 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:43.488742 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:43.488648 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 12:19:43.488742 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:43.488705 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 12:19:43.488813 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:43.488707 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:45.488692183 +0000 UTC m=+50.554284059 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : secret "insights-runtime-extractor-tls" not found Aug 05 12:19:43.488813 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:43.488774 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:45.48875687 +0000 UTC m=+50.554348760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : configmap "kube-rbac-proxy" not found Aug 05 12:19:43.999489 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:43.999462 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:44.542973 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:44.542927 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j7c8w_00e242a0-2726-4da2-b46e-044a68d05c5d/dns-node-resolver/0.log" Aug 05 12:19:45.505779 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:45.505750 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:45.506175 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:45.505792 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:45.506175 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:45.505858 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 12:19:45.506175 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:45.505885 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 12:19:45.506175 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:45.505904 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:49.505891801 +0000 UTC m=+54.571483678 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : configmap "kube-rbac-proxy" not found Aug 05 12:19:45.506175 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:45.505932 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:49.505919487 +0000 UTC m=+54.571511363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : secret "insights-runtime-extractor-tls" not found Aug 05 12:19:45.541789 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:45.541769 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p286z_b8d4ced4-b21e-4a12-8a35-269ad385fcc6/node-ca/0.log" Aug 05 12:19:46.211043 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.211009 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:19:46.211043 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.211051 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:19:46.211043 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.211072 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211160 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211179 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211214 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls podName:5cc3ea90-8123-4429-a46e-ae3f0ab302c1 nodeName:}" failed. No retries permitted until 2026-08-05 12:20:02.211201448 +0000 UTC m=+67.276793325 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls") pod "dns-default-wlqgr" (UID: "5cc3ea90-8123-4429-a46e-ae3f0ab302c1") : secret "dns-default-metrics-tls" not found Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.211212 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211255 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert podName:eb4fa7e4-db72-4734-8a4b-2a55cc102395 nodeName:}" failed. No retries permitted until 2026-08-05 12:20:02.211237481 +0000 UTC m=+67.276829358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-xhr85" (UID: "eb4fa7e4-db72-4734-8a4b-2a55cc102395") : secret "networking-console-plugin-cert" not found Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211184 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211283 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d86dcddf4-dgvbt: secret "image-registry-tls" not found Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.211347 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211358 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls podName:2739193d-8b60-4f26-9d22-f27389fcaab8 nodeName:}" failed. No retries permitted until 2026-08-05 12:20:02.211343712 +0000 UTC m=+67.276935610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls") pod "image-registry-5d86dcddf4-dgvbt" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8") : secret "image-registry-tls" not found Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211371 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211381 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:20:02.211370773 +0000 UTC m=+67.276962660 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : configmap references non-existent config key: service-ca.crt Aug 05 12:19:46.211419 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211412 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls podName:b97663a6-9d83-437a-b7c7-13f5ab0445ae nodeName:}" failed. No retries permitted until 2026-08-05 12:20:02.211397938 +0000 UTC m=+67.276989818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-swmv4" (UID: "b97663a6-9d83-437a-b7c7-13f5ab0445ae") : secret "cluster-monitoring-operator-tls" not found Aug 05 12:19:46.211837 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.211446 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:19:46.211837 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.211472 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:19:46.211837 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211540 2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 05 12:19:46.211837 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211577 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs podName:d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e nodeName:}" failed. No retries permitted until 2026-08-05 12:20:02.211568214 +0000 UTC m=+67.277160093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs") pod "router-default-7cc769bc88-2vr74" (UID: "d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e") : secret "router-metrics-certs-default" not found Aug 05 12:19:46.211837 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211542 2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 12:19:46.211837 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:46.211603 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls podName:cdd81e42-4928-48b5-9e19-296cb4774bef nodeName:}" failed. No retries permitted until 2026-08-05 12:20:02.211597615 +0000 UTC m=+67.277189491 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-fxmhb" (UID: "cdd81e42-4928-48b5-9e19-296cb4774bef") : secret "samples-operator-tls" not found Aug 05 12:19:46.265967 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.265937 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:46.543646 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.543622 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jtxvw_f71fe19e-7bc3-497c-a52e-f21eee789121/serve-healthcheck-canary/0.log" Aug 05 12:19:46.983649 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:46.983580 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:47.143440 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:47.143411 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-s8grw_1e3ef6c1-6fc9-4467-ae88-65c7b7b24436/kube-storage-version-migrator-operator/0.log" Aug 05 12:19:47.676183 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:47.676155 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:48.058305 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:48.058275 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:48.162246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:48.162221 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:48.255385 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:48.255362 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:48.410191 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:48.410131 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:49.405594 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:49.405565 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:49.541563 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:49.541535 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:49.541698 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:49.541579 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:49.541698 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:49.541677 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 12:19:49.541766 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:49.541736 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls podName:ab686670-c502-4b04-aaad-6cb41a462119 nodeName:}" failed. No retries permitted until 2026-08-05 12:19:57.541720162 +0000 UTC m=+62.607312043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shpvc" (UID: "ab686670-c502-4b04-aaad-6cb41a462119") : secret "insights-runtime-extractor-tls" not found Aug 05 12:19:49.542167 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:49.542150 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab686670-c502-4b04-aaad-6cb41a462119-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:49.613741 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:49.613716 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:49.753016 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:49.752941 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:49.753270 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:49.753257 2569 scope.go:117] "RemoveContainer" containerID="2f7f68b5445cbd926aff3423cd61c308b57634e6df6877d1817773916198c5ee" Aug 05 12:19:49.753408 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:49.753391 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-z9wlp_openshift-console-operator(bbc43ec4-08bb-4c69-b4e2-94b3e60eb797)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" podUID="bbc43ec4-08bb-4c69-b4e2-94b3e60eb797" Aug 05 12:19:50.004528 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:50.004448 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:50.212410 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:50.212378 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:50.674716 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:50.674686 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:19:50.675093 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:50.675020 2569 scope.go:117] "RemoveContainer" containerID="2f7f68b5445cbd926aff3423cd61c308b57634e6df6877d1817773916198c5ee" Aug 05 12:19:50.675182 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:19:50.675162 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-z9wlp_openshift-console-operator(bbc43ec4-08bb-4c69-b4e2-94b3e60eb797)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" podUID="bbc43ec4-08bb-4c69-b4e2-94b3e60eb797" Aug 05 12:19:50.813614 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:50.813583 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:51.117366 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:51.117332 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:51.219430 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:51.219403 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:51.386048 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:51.385990 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:51.612293 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:51.612265 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:52.026545 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:52.026513 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:52.125398 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:52.125373 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:52.415329 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:52.415254 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:52.610969 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:52.610922 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:53.044269 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:53.044235 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:53.148966 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:53.148934 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:53.248883 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:53.248856 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:53.823847 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:53.823819 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:53.907167 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:53.907141 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:54.012721 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:54.012699 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:54.115609 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:54.115542 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:54.440099 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:54.440026 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:54.713374 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:54.713312 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:54.840547 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:54.840522 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:55.672435 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:55.672406 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k2j42" Aug 05 12:19:55.825869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:55.825804 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:56.006941 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:56.006911 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:56.237782 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:56.237755 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:56.421717 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:56.421655 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:56.625256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:56.625229 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:57.027098 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:57.027068 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:57.612698 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:57.612670 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:57.614854 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:57.614834 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab686670-c502-4b04-aaad-6cb41a462119-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shpvc\" (UID: \"ab686670-c502-4b04-aaad-6cb41a462119\") " pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:57.681506 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:57.681485 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f4rtr\"" Aug 05 12:19:57.689219 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:57.689202 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-shpvc" Aug 05 12:19:57.770663 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:57.770638 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:57.820359 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:57.820338 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-shpvc"] Aug 05 12:19:57.822503 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:19:57.822473 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab686670_c502_4b04_aaad_6cb41a462119.slice/crio-05cbdb40f3bad6d2304fddb83414aac49b25d04981f851e2cfa684ab9efa592c WatchSource:0}: Error finding container 05cbdb40f3bad6d2304fddb83414aac49b25d04981f851e2cfa684ab9efa592c: Status 404 returned error can't find the container with id 05cbdb40f3bad6d2304fddb83414aac49b25d04981f851e2cfa684ab9efa592c Aug 05 12:19:58.037439 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:58.037408 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:58.219982 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:58.219894 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:58.567107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:58.567063 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:19:58.829764 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:58.829697 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shpvc" event={"ID":"ab686670-c502-4b04-aaad-6cb41a462119","Type":"ContainerStarted","Data":"4094e51112301c2269557cca01ff4f9e38e9a52a19d3919ef50febbbb383503b"} Aug 05 12:19:58.829857 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:58.829779 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shpvc" event={"ID":"ab686670-c502-4b04-aaad-6cb41a462119","Type":"ContainerStarted","Data":"242dbe815b9a9a62fd718f315d54e19c996c392ce5e007bad1491a46a501e12e"} Aug 05 12:19:58.829857 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:19:58.829798 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shpvc" event={"ID":"ab686670-c502-4b04-aaad-6cb41a462119","Type":"ContainerStarted","Data":"05cbdb40f3bad6d2304fddb83414aac49b25d04981f851e2cfa684ab9efa592c"} Aug 05 12:20:00.838207 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:00.838176 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shpvc" event={"ID":"ab686670-c502-4b04-aaad-6cb41a462119","Type":"ContainerStarted","Data":"304b5667e8137959e4609b2a56103d943a191d24a2a4586254f06d39c0353f39"} Aug 05 12:20:00.859602 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:00.859558 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-shpvc" podStartSLOduration=17.525954527 podStartE2EDuration="19.859544273s" totalImagesPullingTime="2.333589746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:19:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:00.858654787 +0000 UTC m=+65.924246686" watchObservedRunningTime="2026-08-05 12:20:00.859544273 +0000 UTC m=+65.925136170" Aug 05 12:20:01.042367 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.042336 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:20:01.042526 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.042375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:20:01.044829 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.044798 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prb2h\" (UniqueName: \"kubernetes.io/projected/bb6cf0be-ed34-4473-b64d-d12bf3212eae-kube-api-access-prb2h\") pod \"network-check-target-z29nk\" (UID: \"bb6cf0be-ed34-4473-b64d-d12bf3212eae\") " pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:20:01.044937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.044856 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 05 12:20:01.054603 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.054567 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccda255-92ed-4f4c-b52f-f3e7c6a639bf-metrics-certs\") pod \"network-metrics-daemon-vsh45\" (UID: \"bccda255-92ed-4f4c-b52f-f3e7c6a639bf\") " pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:20:01.062768 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.062746 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rdzzt\"" Aug 05 12:20:01.071507 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.071486 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:20:01.190483 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.190451 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z29nk"] Aug 05 12:20:01.192970 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:01.192929 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb6cf0be_ed34_4473_b64d_d12bf3212eae.slice/crio-12ea5a9d127eed0df258c8dcec26219dfdda2b5d5b595823fd02d6ab74d881a6 WatchSource:0}: Error finding container 12ea5a9d127eed0df258c8dcec26219dfdda2b5d5b595823fd02d6ab74d881a6: Status 404 returned error can't find the container with id 12ea5a9d127eed0df258c8dcec26219dfdda2b5d5b595823fd02d6ab74d881a6 Aug 05 12:20:01.243678 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.243658 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:20:01.245905 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.245887 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 05 12:20:01.255875 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.255855 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a352b881-3036-4ed7-971a-22645878ebfd-original-pull-secret\") pod \"global-pull-secret-syncer-c6j9w\" (UID: \"a352b881-3036-4ed7-971a-22645878ebfd\") " pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:20:01.350263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.350240 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zm7jq\"" Aug 05 12:20:01.355083 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.355068 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c6j9w" Aug 05 12:20:01.358700 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.358683 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsh45" Aug 05 12:20:01.484816 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.484773 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c6j9w"] Aug 05 12:20:01.488032 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:01.488005 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda352b881_3036_4ed7_971a_22645878ebfd.slice/crio-35fc9b83c0f06f6daa6364c14a69ecbf0db6efe8437af1f51e75d5a27a6aa419 WatchSource:0}: Error finding container 35fc9b83c0f06f6daa6364c14a69ecbf0db6efe8437af1f51e75d5a27a6aa419: Status 404 returned error can't find the container with id 35fc9b83c0f06f6daa6364c14a69ecbf0db6efe8437af1f51e75d5a27a6aa419 Aug 05 12:20:01.500592 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.500570 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vsh45"] Aug 05 12:20:01.503299 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:01.503266 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbccda255_92ed_4f4c_b52f_f3e7c6a639bf.slice/crio-73f93fe3836ef4843e44dd7d279e4160c6e124a6a580abff3cd29b1f88c54873 WatchSource:0}: Error finding container 73f93fe3836ef4843e44dd7d279e4160c6e124a6a580abff3cd29b1f88c54873: Status 404 returned error can't find the container with id 73f93fe3836ef4843e44dd7d279e4160c6e124a6a580abff3cd29b1f88c54873 Aug 05 12:20:01.842768 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.842733 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vsh45" event={"ID":"bccda255-92ed-4f4c-b52f-f3e7c6a639bf","Type":"ContainerStarted","Data":"73f93fe3836ef4843e44dd7d279e4160c6e124a6a580abff3cd29b1f88c54873"} Aug 05 12:20:01.843922 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.843898 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c6j9w" event={"ID":"a352b881-3036-4ed7-971a-22645878ebfd","Type":"ContainerStarted","Data":"35fc9b83c0f06f6daa6364c14a69ecbf0db6efe8437af1f51e75d5a27a6aa419"} Aug 05 12:20:01.845310 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.845287 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z29nk" event={"ID":"bb6cf0be-ed34-4473-b64d-d12bf3212eae","Type":"ContainerStarted","Data":"6dee4314b72424549981e80da10c25f8625f602fbb0466e4490d5bdb6a092bc0"} Aug 05 12:20:01.845419 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.845327 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z29nk" event={"ID":"bb6cf0be-ed34-4473-b64d-d12bf3212eae","Type":"ContainerStarted","Data":"12ea5a9d127eed0df258c8dcec26219dfdda2b5d5b595823fd02d6ab74d881a6"} Aug 05 12:20:01.867281 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:01.867232 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z29nk" podStartSLOduration=66.867215957 podStartE2EDuration="1m6.867215957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:01.86610205 +0000 UTC m=+66.931693949" watchObservedRunningTime="2026-08-05 12:20:01.867215957 +0000 UTC m=+66.932807855" Aug 05 12:20:02.255467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.252020 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:02.255467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.252119 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:20:02.255467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.252158 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:20:02.255467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.252197 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:20:02.255467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.252236 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:02.255467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.252288 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:20:02.255467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.252335 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:20:02.256027 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.255473 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdd81e42-4928-48b5-9e19-296cb4774bef-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-fxmhb\" (UID: \"cdd81e42-4928-48b5-9e19-296cb4774bef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:20:02.256676 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.256244 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-service-ca-bundle\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:02.258015 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.257991 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cc3ea90-8123-4429-a46e-ae3f0ab302c1-metrics-tls\") pod \"dns-default-wlqgr\" (UID: \"5cc3ea90-8123-4429-a46e-ae3f0ab302c1\") " pod="openshift-dns/dns-default-wlqgr" Aug 05 12:20:02.258216 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.258196 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b97663a6-9d83-437a-b7c7-13f5ab0445ae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-swmv4\" (UID: \"b97663a6-9d83-437a-b7c7-13f5ab0445ae\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:20:02.258289 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.258229 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"image-registry-5d86dcddf4-dgvbt\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:20:02.258289 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.258263 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e-metrics-certs\") pod \"router-default-7cc769bc88-2vr74\" (UID: \"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e\") " pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:02.258389 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.258317 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eb4fa7e4-db72-4734-8a4b-2a55cc102395-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-xhr85\" (UID: \"eb4fa7e4-db72-4734-8a4b-2a55cc102395\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:20:02.307942 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.307912 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:02.404414 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.404388 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vkpqd\"" Aug 05 12:20:02.412876 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.412856 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:20:02.413155 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.413140 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6b8rw\"" Aug 05 12:20:02.420800 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.420780 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" Aug 05 12:20:02.423233 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.423215 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:02.431812 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.431797 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-x65bl\"" Aug 05 12:20:02.439699 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.439679 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" Aug 05 12:20:02.449918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.449899 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ftpxl\"" Aug 05 12:20:02.458972 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.458934 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" Aug 05 12:20:02.468640 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.468618 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4c62j\"" Aug 05 12:20:02.477350 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.477325 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wlqgr" Aug 05 12:20:02.496084 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.496059 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5l78w\"" Aug 05 12:20:02.503746 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.503722 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:02.593588 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.593373 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d86dcddf4-dgvbt"] Aug 05 12:20:02.643564 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.643332 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4"] Aug 05 12:20:02.720208 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.720123 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb"] Aug 05 12:20:02.722144 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.721545 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:02.750065 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.750000 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-xhr85"] Aug 05 12:20:02.758988 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:02.758886 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4fa7e4_db72_4734_8a4b_2a55cc102395.slice/crio-3863d46b0aa7d45ae25ae7c220d08fc65158eb71f46f3ca753fac01b138edd4e WatchSource:0}: Error finding container 3863d46b0aa7d45ae25ae7c220d08fc65158eb71f46f3ca753fac01b138edd4e: Status 404 returned error can't find the container with id 3863d46b0aa7d45ae25ae7c220d08fc65158eb71f46f3ca753fac01b138edd4e Aug 05 12:20:02.794535 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.794376 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cc769bc88-2vr74"] Aug 05 12:20:02.797419 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.797325 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:02.800666 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:02.800641 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd879f9c0_8e83_4585_b3f9_9b1f72fc3e4e.slice/crio-e6f526c67d7e875e459591ef58d9bafc76cc006969b55f24fc5f57b3a2849f53 WatchSource:0}: Error finding container e6f526c67d7e875e459591ef58d9bafc76cc006969b55f24fc5f57b3a2849f53: Status 404 returned error can't find the container with id e6f526c67d7e875e459591ef58d9bafc76cc006969b55f24fc5f57b3a2849f53 Aug 05 12:20:02.818234 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.818200 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wlqgr"] Aug 05 12:20:02.823138 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:02.823098 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc3ea90_8123_4429_a46e_ae3f0ab302c1.slice/crio-951763ca8aea98e2d94ab9a753b384819b75ead0cbf232e80633fab896a3d2b5 WatchSource:0}: Error finding container 951763ca8aea98e2d94ab9a753b384819b75ead0cbf232e80633fab896a3d2b5: Status 404 returned error can't find the container with id 951763ca8aea98e2d94ab9a753b384819b75ead0cbf232e80633fab896a3d2b5 Aug 05 12:20:02.850769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.850693 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" event={"ID":"cdd81e42-4928-48b5-9e19-296cb4774bef","Type":"ContainerStarted","Data":"8289cec18d5c26489849495cbea4e9d5bb3aac9b07c729e713bfab386e47f87a"} Aug 05 12:20:02.852551 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.852495 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" event={"ID":"eb4fa7e4-db72-4734-8a4b-2a55cc102395","Type":"ContainerStarted","Data":"3863d46b0aa7d45ae25ae7c220d08fc65158eb71f46f3ca753fac01b138edd4e"} Aug 05 12:20:02.854360 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.854323 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" event={"ID":"b97663a6-9d83-437a-b7c7-13f5ab0445ae","Type":"ContainerStarted","Data":"8d91b60a37724d9a741d023e9e300827e97170cc2aae81f528437960a880f795"} Aug 05 12:20:02.856853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.856830 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" event={"ID":"2739193d-8b60-4f26-9d22-f27389fcaab8","Type":"ContainerStarted","Data":"3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8"} Aug 05 12:20:02.856978 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.856871 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" event={"ID":"2739193d-8b60-4f26-9d22-f27389fcaab8","Type":"ContainerStarted","Data":"cf0c02d45c985e4ad117778cbda3b9d449851651f2e04d7707b73e7ade2f854e"} Aug 05 12:20:02.857468 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.857449 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:20:02.858697 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.858673 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wlqgr" event={"ID":"5cc3ea90-8123-4429-a46e-ae3f0ab302c1","Type":"ContainerStarted","Data":"951763ca8aea98e2d94ab9a753b384819b75ead0cbf232e80633fab896a3d2b5"} Aug 05 12:20:02.860158 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.860136 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cc769bc88-2vr74" event={"ID":"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e","Type":"ContainerStarted","Data":"e6f526c67d7e875e459591ef58d9bafc76cc006969b55f24fc5f57b3a2849f53"} Aug 05 12:20:02.860250 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.860179 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:20:02.879221 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:02.879170 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" podStartSLOduration=66.879154952 podStartE2EDuration="1m6.879154952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:02.878445168 +0000 UTC m=+67.944037063" watchObservedRunningTime="2026-08-05 12:20:02.879154952 +0000 UTC m=+67.944746851" Aug 05 12:20:03.868978 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:03.868546 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cc769bc88-2vr74" event={"ID":"d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e","Type":"ContainerStarted","Data":"302767014dd15fa8fa8abfebb0a5fed064fcfd6920b6eaa85bbe38dc9ddff99f"} Aug 05 12:20:03.877692 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:03.877664 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vsh45" event={"ID":"bccda255-92ed-4f4c-b52f-f3e7c6a639bf","Type":"ContainerStarted","Data":"a261569b3180647ef198e427a04e1a240c42d0d5b4b901d1b76a6708877ddd9a"} Aug 05 12:20:03.878359 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:03.878338 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vsh45" event={"ID":"bccda255-92ed-4f4c-b52f-f3e7c6a639bf","Type":"ContainerStarted","Data":"92e06928da5a65542120313474886589ae571cdcaafb71a461ce78ee9696482d"} Aug 05 12:20:03.892033 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:03.890068 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7cc769bc88-2vr74" podStartSLOduration=66.890048521 podStartE2EDuration="1m6.890048521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:03.888913136 +0000 UTC m=+68.954505028" watchObservedRunningTime="2026-08-05 12:20:03.890048521 +0000 UTC m=+68.955640419" Aug 05 12:20:03.904535 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:03.903487 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vsh45" podStartSLOduration=66.98959153 podStartE2EDuration="1m8.903469136s" totalImagesPullingTime="1.913877606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:03.902842684 +0000 UTC m=+68.968434582" watchObservedRunningTime="2026-08-05 12:20:03.903469136 +0000 UTC m=+68.969061035" Aug 05 12:20:04.015443 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.015359 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:04.077104 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.077071 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:04.436911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.436843 2569 scope.go:117] "RemoveContainer" containerID="2f7f68b5445cbd926aff3423cd61c308b57634e6df6877d1817773916198c5ee" Aug 05 12:20:04.504177 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.504100 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:04.509201 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.508903 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:04.885211 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.884097 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:20:04.888473 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.885774 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" event={"ID":"bbc43ec4-08bb-4c69-b4e2-94b3e60eb797","Type":"ContainerStarted","Data":"c0dfba0eaa0df0a52b59622d49dc49e24591f446d767cd9ac9dc7d0cd2b9db47"} Aug 05 12:20:04.888473 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.885870 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:04.888473 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.887998 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:20:04.888869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:04.888812 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7cc769bc88-2vr74" Aug 05 12:20:05.033977 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.033864 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-z9wlp" Aug 05 12:20:05.226741 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.226614 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:05.285583 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.285532 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:05.391023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.390991 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:05.509944 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.509909 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:05.610340 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.610310 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:05.712232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.712195 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:05.772769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.772698 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:05.947207 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:05.947170 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:06.004099 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.004072 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:06.124470 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.124394 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:06.208904 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.208872 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:06.524202 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.524171 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:06.592854 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.592815 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:06.712643 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.712610 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:06.859794 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.859714 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:06.894131 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.894095 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" event={"ID":"eb4fa7e4-db72-4734-8a4b-2a55cc102395","Type":"ContainerStarted","Data":"3fbbdb6c733874b2e00a5ac33a89df53fb61ade44ea27f8dbe8ffaf4fd3de8df"} Aug 05 12:20:06.910642 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:06.910594 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-xhr85" podStartSLOduration=59.044155993 podStartE2EDuration="1m0.910575047s" totalImagesPullingTime="1.866419054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:06.909252923 +0000 UTC m=+71.974844822" watchObservedRunningTime="2026-08-05 12:20:06.910575047 +0000 UTC m=+71.976166947" Aug 05 12:20:07.022486 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.022445 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:07.127096 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.127020 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:07.326312 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.326259 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:07.418008 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.417910 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:07.901540 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.901510 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" event={"ID":"cdd81e42-4928-48b5-9e19-296cb4774bef","Type":"ContainerStarted","Data":"e0db323c50150d78a3a1471756c875064c2495f99efb43200023ed3cf05f5866"} Aug 05 12:20:07.901678 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.901558 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" event={"ID":"cdd81e42-4928-48b5-9e19-296cb4774bef","Type":"ContainerStarted","Data":"7c1f70290a82dd30d5ecc62c0ee17fa26467f63388990dddf3ad81faacb30471"} Aug 05 12:20:07.903911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.903856 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" event={"ID":"b97663a6-9d83-437a-b7c7-13f5ab0445ae","Type":"ContainerStarted","Data":"4f35808cf285a1c129e75e9a77eb9dfe0f9216f2ff62ad79da9f385eaee84343"} Aug 05 12:20:07.905670 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.905649 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wlqgr" event={"ID":"5cc3ea90-8123-4429-a46e-ae3f0ab302c1","Type":"ContainerStarted","Data":"27b93f9ee8938946682138c382855274700ebd1eca350317c6d98d4794eac2e7"} Aug 05 12:20:07.908852 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.908180 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c6j9w" event={"ID":"a352b881-3036-4ed7-971a-22645878ebfd","Type":"ContainerStarted","Data":"2b826fbd3908935d01fae9da934fd8dee42eec2a93a09a8267e7f6afc3c5af9c"} Aug 05 12:20:07.924527 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.924442 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-fxmhb" podStartSLOduration=66.060095029 podStartE2EDuration="1m10.924424004s" totalImagesPullingTime="4.864328975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:07.923308138 +0000 UTC m=+72.988900062" watchObservedRunningTime="2026-08-05 12:20:07.924424004 +0000 UTC m=+72.990015904" Aug 05 12:20:07.943772 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.942925 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c6j9w" podStartSLOduration=65.745526895 podStartE2EDuration="1m11.942906705s" totalImagesPullingTime="6.19737981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:07.940445156 +0000 UTC m=+73.006037049" watchObservedRunningTime="2026-08-05 12:20:07.942906705 +0000 UTC m=+73.008498604" Aug 05 12:20:07.959197 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:07.959025 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-swmv4" podStartSLOduration=65.953781934 podStartE2EDuration="1m10.959008015s" totalImagesPullingTime="5.005226081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:07.958107007 +0000 UTC m=+73.023698905" watchObservedRunningTime="2026-08-05 12:20:07.959008015 +0000 UTC m=+73.024599916" Aug 05 12:20:08.120832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:08.120794 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:08.240639 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:08.240569 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:08.316221 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:08.316167 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:08.623341 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:08.623308 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:08.913414 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:08.913316 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wlqgr" event={"ID":"5cc3ea90-8123-4429-a46e-ae3f0ab302c1","Type":"ContainerStarted","Data":"5d7914ebe71540d63e4558917978fb91672d3e70594a4e8acc5cc8410624a2cb"} Aug 05 12:20:08.913903 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:08.913876 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wlqgr" Aug 05 12:20:08.930444 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:08.930317 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wlqgr" podStartSLOduration=34.080989149 podStartE2EDuration="38.93029921s" totalImagesPullingTime="4.849310061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:08.930011269 +0000 UTC m=+73.995603168" watchObservedRunningTime="2026-08-05 12:20:08.93029921 +0000 UTC m=+73.995891110" Aug 05 12:20:09.322038 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:09.322007 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:09.381943 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:09.381917 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:11.718227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:11.718195 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:11.817061 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:11.817032 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:13.327131 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:13.327100 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:13.401660 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:13.401631 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:13.729896 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:13.729818 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:14.000426 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:14.000361 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:16.129391 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:16.129358 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:16.524395 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:16.524360 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:16.625090 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:16.625059 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:18.465343 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.465242 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:18.684370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.684338 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:18.753564 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.753488 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj"] Aug 05 12:20:18.756547 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.756527 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.759656 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.759638 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Aug 05 12:20:18.760163 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.760147 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Aug 05 12:20:18.760707 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.760691 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 05 12:20:18.761404 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.761387 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rghgp\"" Aug 05 12:20:18.761482 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.761428 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Aug 05 12:20:18.763639 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.763505 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8lmkl"] Aug 05 12:20:18.765912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.765895 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.768134 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.768116 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 05 12:20:18.768203 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.768185 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dwmqg\"" Aug 05 12:20:18.768203 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.768192 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 05 12:20:18.768310 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.768298 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 05 12:20:18.785932 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.785903 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj"] Aug 05 12:20:18.816481 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.816460 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:18.883278 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-wtmp\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.883382 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883282 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-textfile\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.883382 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883300 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-tls\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.883382 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883318 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c70ebb7b-1a52-4f4b-b8f2-430938d63054-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.883382 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883363 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.883515 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883389 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4pgx\" (UniqueName: \"kubernetes.io/projected/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-api-access-b4pgx\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.883515 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883429 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-root\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.883515 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883470 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.883515 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883500 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-sys\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.883626 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883532 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-metrics-client-ca\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.883626 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883552 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.883626 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kxbq\" (UniqueName: \"kubernetes.io/projected/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-kube-api-access-8kxbq\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.883626 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883599 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.883626 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883619 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c70ebb7b-1a52-4f4b-b8f2-430938d63054-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.883771 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.883671 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.900749 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.900582 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:18.984456 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984430 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-metrics-client-ca\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.984562 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984459 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.984562 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984483 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kxbq\" (UniqueName: \"kubernetes.io/projected/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-kube-api-access-8kxbq\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.984562 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984506 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.984679 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984631 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c70ebb7b-1a52-4f4b-b8f2-430938d63054-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.984723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984686 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.984759 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984728 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-wtmp\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.984798 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984756 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-textfile\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.984798 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984778 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-tls\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.984902 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984803 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c70ebb7b-1a52-4f4b-b8f2-430938d63054-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.984902 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.984838 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.985075 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:20:18.985059 2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 05 12:20:18.985134 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:20:18.985120 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-tls podName:9b56f15e-126a-4abc-bdd0-3c48cd3ba988 nodeName:}" failed. No retries permitted until 2026-08-05 12:20:19.485101537 +0000 UTC m=+84.550693428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-tls") pod "node-exporter-8lmkl" (UID: "9b56f15e-126a-4abc-bdd0-3c48cd3ba988") : secret "node-exporter-tls" not found Aug 05 12:20:18.985233 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985211 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-metrics-client-ca\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.985369 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985211 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-accelerators-collector-config\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.985464 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985237 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-textfile\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.985569 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985263 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4pgx\" (UniqueName: \"kubernetes.io/projected/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-api-access-b4pgx\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.985661 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985601 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-root\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.985661 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985413 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-wtmp\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.985661 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985452 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c70ebb7b-1a52-4f4b-b8f2-430938d63054-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.985661 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985491 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c70ebb7b-1a52-4f4b-b8f2-430938d63054-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.985848 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985655 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.985848 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985680 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-root\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.985848 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985697 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-sys\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.985848 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985787 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-sys\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.986028 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.985920 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.987105 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.987086 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.987192 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.987152 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.987807 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.987790 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:18.993513 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.993493 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kxbq\" (UniqueName: \"kubernetes.io/projected/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-kube-api-access-8kxbq\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:18.993846 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:18.993826 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4pgx\" (UniqueName: \"kubernetes.io/projected/c70ebb7b-1a52-4f4b-b8f2-430938d63054-kube-api-access-b4pgx\") pod \"kube-state-metrics-6bd9664c4-d69mj\" (UID: \"c70ebb7b-1a52-4f4b-b8f2-430938d63054\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:19.005377 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.005321 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:19.066453 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.066433 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" Aug 05 12:20:19.101678 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.101645 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:19.202340 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.202199 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj"] Aug 05 12:20:19.204887 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:19.204855 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70ebb7b_1a52_4f4b_b8f2_430938d63054.slice/crio-6ed3014d2603be01973a5300a2ef6e843f9c3f9bbb0024beabc6fead38ac2547 WatchSource:0}: Error finding container 6ed3014d2603be01973a5300a2ef6e843f9c3f9bbb0024beabc6fead38ac2547: Status 404 returned error can't find the container with id 6ed3014d2603be01973a5300a2ef6e843f9c3f9bbb0024beabc6fead38ac2547 Aug 05 12:20:19.205658 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.205576 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:19.306880 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.306855 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:19.490652 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.490624 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-tls\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:19.492777 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.492758 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9b56f15e-126a-4abc-bdd0-3c48cd3ba988-node-exporter-tls\") pod \"node-exporter-8lmkl\" (UID: \"9b56f15e-126a-4abc-bdd0-3c48cd3ba988\") " pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:19.674541 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.674464 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8lmkl" Aug 05 12:20:19.687880 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:19.687852 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b56f15e_126a_4abc_bdd0_3c48cd3ba988.slice/crio-08b7d47e23ea999b2717883b2c20d5e37d6e74b9da85104340b0ee0f4cb0cc04 WatchSource:0}: Error finding container 08b7d47e23ea999b2717883b2c20d5e37d6e74b9da85104340b0ee0f4cb0cc04: Status 404 returned error can't find the container with id 08b7d47e23ea999b2717883b2c20d5e37d6e74b9da85104340b0ee0f4cb0cc04 Aug 05 12:20:19.707498 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.707473 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:19.803003 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.802944 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:19.809914 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.809892 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:20:19.812611 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.812596 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.815111 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815088 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 05 12:20:19.815223 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815140 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 05 12:20:19.815223 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815171 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 05 12:20:19.815223 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815199 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 05 12:20:19.815433 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815419 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 05 12:20:19.815507 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815491 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 05 12:20:19.815507 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815502 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zhkcg\"" Aug 05 12:20:19.815607 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815516 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 05 12:20:19.815607 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815547 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 05 12:20:19.815607 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.815494 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 05 12:20:19.828397 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.828371 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:20:19.925491 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.925421 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wlqgr" Aug 05 12:20:19.956394 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.956310 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lmkl" event={"ID":"9b56f15e-126a-4abc-bdd0-3c48cd3ba988","Type":"ContainerStarted","Data":"08b7d47e23ea999b2717883b2c20d5e37d6e74b9da85104340b0ee0f4cb0cc04"} Aug 05 12:20:19.958422 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.958383 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" event={"ID":"c70ebb7b-1a52-4f4b-b8f2-430938d63054","Type":"ContainerStarted","Data":"6ed3014d2603be01973a5300a2ef6e843f9c3f9bbb0024beabc6fead38ac2547"} Aug 05 12:20:19.994630 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994599 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-web-config\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.994757 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994644 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.994757 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994675 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8czrd\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-kube-api-access-8czrd\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.994870 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994758 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.994870 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994791 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.994870 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994855 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-config-out\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.995053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994895 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.995053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994926 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.995053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994941 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-config-volume\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.995053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994979 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.995053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.994998 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.995053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.995016 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:19.995053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:19.995032 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.090309 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.090273 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:20.095413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095380 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.095536 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095431 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.095536 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095466 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.095536 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095495 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.095708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095620 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-web-config\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.095708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095666 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.095708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095697 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8czrd\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-kube-api-access-8czrd\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.095850 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095767 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.095850 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.095793 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.096256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.096221 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.096338 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:20:20.096321 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle podName:d8929588-cc30-4903-80b2-26c80e4830ce nodeName:}" failed. No retries permitted until 2026-08-05 12:20:20.596298507 +0000 UTC m=+85.661890386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce") : configmap references non-existent config key: ca-bundle.crt Aug 05 12:20:20.096508 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.096486 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.096660 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.096639 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-config-out\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.096765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.096737 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.096850 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.096787 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.096850 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.096818 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-config-volume\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.098967 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.098884 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.099073 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.098990 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.100891 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.100657 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.100891 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.100840 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-config-volume\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.101094 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.101073 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-config-out\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.101926 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.101903 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.102283 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.102241 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-web-config\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.102527 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.102509 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.102721 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.102692 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.108299 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.108274 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8czrd\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-kube-api-access-8czrd\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.602135 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.602098 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.603070 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.603045 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.722782 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.722748 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:20:20.938188 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.937185 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:20:20.969500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.969457 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" event={"ID":"c70ebb7b-1a52-4f4b-b8f2-430938d63054","Type":"ContainerStarted","Data":"a8805404a66db8904f3570dd3ef14d8b9ee1bbd0ded53b40b1260303357e2326"} Aug 05 12:20:20.969635 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.969543 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" event={"ID":"c70ebb7b-1a52-4f4b-b8f2-430938d63054","Type":"ContainerStarted","Data":"39a1ba7de0a76342159adff8723170be83fc0f4dc5ca5dcbed89577dd557b0e3"} Aug 05 12:20:20.971200 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.971169 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerStarted","Data":"e292509f01362cc2aca9af0ee3625675f09b0267fed486cd37c88241bb3f5a15"} Aug 05 12:20:20.973373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:20.973342 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lmkl" event={"ID":"9b56f15e-126a-4abc-bdd0-3c48cd3ba988","Type":"ContainerStarted","Data":"72dbaf2778e6ca780a2758e77e5fc3695d0f70fcf0d56468b864f73f3c1395d8"} Aug 05 12:20:21.295526 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:21.295477 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:21.679466 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:21.679383 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:21.842384 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:21.842348 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:21.982424 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:21.982339 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" event={"ID":"c70ebb7b-1a52-4f4b-b8f2-430938d63054","Type":"ContainerStarted","Data":"a8c3fc42551aa6454189bad43d9f0c1fdb231c6987ae2383dde555f78f9cd7e9"} Aug 05 12:20:21.984038 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:21.984005 2569 generic.go:356] "Generic (PLEG): container finished" podID="9b56f15e-126a-4abc-bdd0-3c48cd3ba988" containerID="72dbaf2778e6ca780a2758e77e5fc3695d0f70fcf0d56468b864f73f3c1395d8" exitCode=0 Aug 05 12:20:21.984155 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:21.984076 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lmkl" event={"ID":"9b56f15e-126a-4abc-bdd0-3c48cd3ba988","Type":"ContainerDied","Data":"72dbaf2778e6ca780a2758e77e5fc3695d0f70fcf0d56468b864f73f3c1395d8"} Aug 05 12:20:22.002307 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.002270 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-d69mj" podStartSLOduration=2.423077043 podStartE2EDuration="4.002254667s" totalImagesPullingTime="1.579177624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:22.000346516 +0000 UTC m=+87.065938440" watchObservedRunningTime="2026-08-05 12:20:22.002254667 +0000 UTC m=+87.067846566" Aug 05 12:20:22.095978 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.095927 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:22.296475 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.296451 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:22.417999 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.417968 2569 patch_prober.go:28] interesting pod/image-registry-5d86dcddf4-dgvbt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Aug 05 12:20:22.418103 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.418016 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" podUID="2739193d-8b60-4f26-9d22-f27389fcaab8" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 05 12:20:22.989054 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.989024 2569 generic.go:356] "Generic (PLEG): container finished" podID="d8929588-cc30-4903-80b2-26c80e4830ce" containerID="24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b" exitCode=0 Aug 05 12:20:22.989468 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.989100 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerDied","Data":"24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b"} Aug 05 12:20:22.991256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.991233 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lmkl" event={"ID":"9b56f15e-126a-4abc-bdd0-3c48cd3ba988","Type":"ContainerStarted","Data":"6888921f298f71724d2fccfa9c44004554c431c7db21bc9f9149f1a8f126ad1f"} Aug 05 12:20:22.991370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:22.991275 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8lmkl" event={"ID":"9b56f15e-126a-4abc-bdd0-3c48cd3ba988","Type":"ContainerStarted","Data":"95000f15e610034b4acf88e79012bffa5fc9c8602401df1e30534265d43c9c2b"} Aug 05 12:20:23.037306 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.037259 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8lmkl" podStartSLOduration=3.8213484319999997 podStartE2EDuration="5.037246317s" totalImagesPullingTime="1.120254777s" totalInitContainerRuntime="95.643108ms" isStatefulPod=true podCreationTimestamp="2026-08-05 12:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:23.03662953 +0000 UTC m=+88.102221443" watchObservedRunningTime="2026-08-05 12:20:23.037246317 +0000 UTC m=+88.102838216" Aug 05 12:20:23.155012 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.154977 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b665b4949-2rlhw"] Aug 05 12:20:23.157174 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.157153 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.159842 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.159820 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 05 12:20:23.160025 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.159994 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Aug 05 12:20:23.160025 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.160019 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dkuruq2rf6c7l\"" Aug 05 12:20:23.160156 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.160026 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Aug 05 12:20:23.160156 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.160117 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dhq9t\"" Aug 05 12:20:23.160252 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.160160 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Aug 05 12:20:23.170418 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.170392 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b665b4949-2rlhw"] Aug 05 12:20:23.224127 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.224101 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-secret-metrics-server-tls\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.224240 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.224134 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a68d94e0-b82d-435d-8cb9-3f0aee82b217-metrics-server-audit-profiles\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.224240 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.224160 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqh4k\" (UniqueName: \"kubernetes.io/projected/a68d94e0-b82d-435d-8cb9-3f0aee82b217-kube-api-access-rqh4k\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.224240 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.224185 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a68d94e0-b82d-435d-8cb9-3f0aee82b217-audit-log\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.224240 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.224230 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-client-ca-bundle\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.224413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.224246 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68d94e0-b82d-435d-8cb9-3f0aee82b217-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.224413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.224273 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-secret-metrics-server-client-certs\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.325235 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.325204 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-client-ca-bundle\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.325361 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.325241 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68d94e0-b82d-435d-8cb9-3f0aee82b217-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.325361 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.325273 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-secret-metrics-server-client-certs\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.325361 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.325313 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-secret-metrics-server-tls\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.325361 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.325345 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a68d94e0-b82d-435d-8cb9-3f0aee82b217-metrics-server-audit-profiles\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.325554 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.325381 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqh4k\" (UniqueName: \"kubernetes.io/projected/a68d94e0-b82d-435d-8cb9-3f0aee82b217-kube-api-access-rqh4k\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.325624 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.325575 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a68d94e0-b82d-435d-8cb9-3f0aee82b217-audit-log\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.325980 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.325919 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a68d94e0-b82d-435d-8cb9-3f0aee82b217-audit-log\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.326099 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.326017 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68d94e0-b82d-435d-8cb9-3f0aee82b217-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.326328 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.326308 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a68d94e0-b82d-435d-8cb9-3f0aee82b217-metrics-server-audit-profiles\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.327837 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.327814 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-secret-metrics-server-client-certs\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.327937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.327880 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-client-ca-bundle\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.327937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.327913 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a68d94e0-b82d-435d-8cb9-3f0aee82b217-secret-metrics-server-tls\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.334076 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.334055 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqh4k\" (UniqueName: \"kubernetes.io/projected/a68d94e0-b82d-435d-8cb9-3f0aee82b217-kube-api-access-rqh4k\") pod \"metrics-server-5b665b4949-2rlhw\" (UID: \"a68d94e0-b82d-435d-8cb9-3f0aee82b217\") " pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.466389 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.466358 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:23.501277 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.501247 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:23.602365 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.602295 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:23.609355 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.609329 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b665b4949-2rlhw"] Aug 05 12:20:23.612225 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:23.612200 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68d94e0_b82d_435d_8cb9_3f0aee82b217.slice/crio-15efa8c7913abca2b868aa4b0822544f3e3650ae08b2fee77faa1eeefffc4f3d WatchSource:0}: Error finding container 15efa8c7913abca2b868aa4b0822544f3e3650ae08b2fee77faa1eeefffc4f3d: Status 404 returned error can't find the container with id 15efa8c7913abca2b868aa4b0822544f3e3650ae08b2fee77faa1eeefffc4f3d Aug 05 12:20:23.701001 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.700969 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:23.803470 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.803444 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:23.997274 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:23.997185 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" event={"ID":"a68d94e0-b82d-435d-8cb9-3f0aee82b217","Type":"ContainerStarted","Data":"15efa8c7913abca2b868aa4b0822544f3e3650ae08b2fee77faa1eeefffc4f3d"} Aug 05 12:20:24.484452 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:24.484429 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d86dcddf4-dgvbt"] Aug 05 12:20:24.490120 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:24.490097 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:20:24.576246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:24.576220 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:24.677309 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:24.677213 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:24.776825 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:24.776786 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:24.945308 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:24.945278 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:25.006291 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:25.006258 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerStarted","Data":"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50"} Aug 05 12:20:25.006691 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:25.006332 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerStarted","Data":"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6"} Aug 05 12:20:25.006691 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:25.006363 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerStarted","Data":"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1"} Aug 05 12:20:25.006691 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:25.006406 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerStarted","Data":"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9"} Aug 05 12:20:25.006691 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:25.006441 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerStarted","Data":"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26"} Aug 05 12:20:25.052145 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:25.052084 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:26.013458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.013417 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerStarted","Data":"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b"} Aug 05 12:20:26.014933 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.014909 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" event={"ID":"a68d94e0-b82d-435d-8cb9-3f0aee82b217","Type":"ContainerStarted","Data":"e9b6b3a2d347bd4f55fb15f03cb6862804afbc206dd7e01ac0fb47fce006a918"} Aug 05 12:20:26.051288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.051229 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.541152403 podStartE2EDuration="7.051214969s" totalImagesPullingTime="3.487235403s" totalInitContainerRuntime="22.827163ms" isStatefulPod=false podCreationTimestamp="2026-08-05 12:20:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:26.051059225 +0000 UTC m=+91.116651124" watchObservedRunningTime="2026-08-05 12:20:26.051214969 +0000 UTC m=+91.116806867" Aug 05 12:20:26.080561 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.080519 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" podStartSLOduration=1.510336811 podStartE2EDuration="3.080507065s" totalImagesPullingTime="1.570170254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:26.078941286 +0000 UTC m=+91.144533177" watchObservedRunningTime="2026-08-05 12:20:26.080507065 +0000 UTC m=+91.146098964" Aug 05 12:20:26.321261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.321185 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7548c6b4c-jsblx"] Aug 05 12:20:26.323341 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.323325 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.325816 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.325772 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Aug 05 12:20:26.325816 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.325800 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Aug 05 12:20:26.326001 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.325858 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Aug 05 12:20:26.326001 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.325909 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Aug 05 12:20:26.326001 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.325946 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 05 12:20:26.326001 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.325978 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7bdrz\"" Aug 05 12:20:26.326138 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.326040 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Aug 05 12:20:26.326337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.326323 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 05 12:20:26.335075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.335050 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Aug 05 12:20:26.336784 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.336761 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7548c6b4c-jsblx"] Aug 05 12:20:26.346106 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.346085 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-console-config\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.346188 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.346114 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-service-ca\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.346188 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.346139 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-serving-cert\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.346268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.346207 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-oauth-serving-cert\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.346268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.346251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8kwb\" (UniqueName: \"kubernetes.io/projected/3a4ec894-3eb2-4564-8208-03a277fad951-kube-api-access-w8kwb\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.346327 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.346304 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-trusted-ca-bundle\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.346360 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.346346 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-oauth-config\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.447196 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.447172 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-serving-cert\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.447328 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.447213 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-oauth-serving-cert\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.447328 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.447229 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8kwb\" (UniqueName: \"kubernetes.io/projected/3a4ec894-3eb2-4564-8208-03a277fad951-kube-api-access-w8kwb\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.447328 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.447269 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-trusted-ca-bundle\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.447328 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.447303 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-oauth-config\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.447467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.447363 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-console-config\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.447467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.447402 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-service-ca\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.448036 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.448015 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-service-ca\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.448134 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.448050 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-oauth-serving-cert\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.448134 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.448111 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-console-config\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.448537 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.448512 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-trusted-ca-bundle\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.449688 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.449665 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-serving-cert\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.449826 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.449805 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-oauth-config\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.454893 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.454875 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8kwb\" (UniqueName: \"kubernetes.io/projected/3a4ec894-3eb2-4564-8208-03a277fad951-kube-api-access-w8kwb\") pod \"console-7548c6b4c-jsblx\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.635373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.635321 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:26.711664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.711628 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:26.766822 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.766547 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7548c6b4c-jsblx"] Aug 05 12:20:26.769711 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:20:26.769682 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4ec894_3eb2_4564_8208_03a277fad951.slice/crio-f1c3d257f22da63b57e5567c67f1c09afbc65c8f19db128e78ac66f07dfa141f WatchSource:0}: Error finding container f1c3d257f22da63b57e5567c67f1c09afbc65c8f19db128e78ac66f07dfa141f: Status 404 returned error can't find the container with id f1c3d257f22da63b57e5567c67f1c09afbc65c8f19db128e78ac66f07dfa141f Aug 05 12:20:26.848668 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:26.848643 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:27.020107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:27.020074 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7548c6b4c-jsblx" event={"ID":"3a4ec894-3eb2-4564-8208-03a277fad951","Type":"ContainerStarted","Data":"f1c3d257f22da63b57e5567c67f1c09afbc65c8f19db128e78ac66f07dfa141f"} Aug 05 12:20:27.923753 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:27.923696 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:28.022794 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:28.022754 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:29.064189 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:29.064152 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:29.162007 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:29.161972 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:30.035258 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:30.035227 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7548c6b4c-jsblx" event={"ID":"3a4ec894-3eb2-4564-8208-03a277fad951","Type":"ContainerStarted","Data":"2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d"} Aug 05 12:20:30.053844 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:30.053780 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7548c6b4c-jsblx" podStartSLOduration=1.3797883610000001 podStartE2EDuration="4.053760859s" totalImagesPullingTime="2.673972498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:20:30.052892072 +0000 UTC m=+95.118483971" watchObservedRunningTime="2026-08-05 12:20:30.053760859 +0000 UTC m=+95.119352758" Aug 05 12:20:31.184088 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:31.184056 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:31.284620 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:31.284595 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:32.330442 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:32.330399 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:32.433878 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:32.433848 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:33.879843 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:33.879810 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z29nk" Aug 05 12:20:34.128878 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:34.128837 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:34.232703 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:34.232620 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:36.636208 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:36.636172 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:36.636554 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:36.636257 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:36.641045 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:36.641023 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:37.065926 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:37.065902 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:20:37.551085 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:37.551051 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:37.648116 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:37.648089 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:38.092494 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:38.092464 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:38.190084 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:38.190056 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:38.531206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:38.531175 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:40.259415 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:40.259377 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:40.421296 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:40.421254 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:41.718628 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:41.718599 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:41.820988 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:41.820945 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:42.030470 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:42.030442 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:42.130337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:42.130307 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:43.466666 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:43.466634 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:43.467138 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:43.466690 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:20:46.096232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:46.096198 2569 generic.go:356] "Generic (PLEG): container finished" podID="4352fed2-2dde-4ff6-9cb7-998b3fded2e7" containerID="2b40168a1462703c041362f6e363328476f823849927adbd6196f3fd82f57022" exitCode=0 Aug 05 12:20:46.096626 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:46.096268 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" event={"ID":"4352fed2-2dde-4ff6-9cb7-998b3fded2e7","Type":"ContainerDied","Data":"2b40168a1462703c041362f6e363328476f823849927adbd6196f3fd82f57022"} Aug 05 12:20:46.096697 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:46.096640 2569 scope.go:117] "RemoveContainer" containerID="2b40168a1462703c041362f6e363328476f823849927adbd6196f3fd82f57022" Aug 05 12:20:46.097766 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:46.097745 2569 generic.go:356] "Generic (PLEG): container finished" podID="dd67cf94-77ba-4851-8abb-2981237ae617" containerID="150ccd40cb64660e5f9a7d6ae53ac82e11959e15880298681e82d7c12bfcf70c" exitCode=0 Aug 05 12:20:46.097883 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:46.097806 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-7qgvq" event={"ID":"dd67cf94-77ba-4851-8abb-2981237ae617","Type":"ContainerDied","Data":"150ccd40cb64660e5f9a7d6ae53ac82e11959e15880298681e82d7c12bfcf70c"} Aug 05 12:20:46.098098 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:46.098082 2569 scope.go:117] "RemoveContainer" containerID="150ccd40cb64660e5f9a7d6ae53ac82e11959e15880298681e82d7c12bfcf70c" Aug 05 12:20:46.949311 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:46.949281 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cc769bc88-2vr74_d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e/router/0.log" Aug 05 12:20:46.972199 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:46.972173 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jtxvw_f71fe19e-7bc3-497c-a52e-f21eee789121/serve-healthcheck-canary/0.log" Aug 05 12:20:47.104347 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:47.104312 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-lxj6z" event={"ID":"4352fed2-2dde-4ff6-9cb7-998b3fded2e7","Type":"ContainerStarted","Data":"0d372f26d99569ac70cfb29a16e58d2f424643bbe52522cba8ff0b007e86ea6d"} Aug 05 12:20:47.106290 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:47.106267 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-7qgvq" event={"ID":"dd67cf94-77ba-4851-8abb-2981237ae617","Type":"ContainerStarted","Data":"17f795fd0fad703789490a170134a2a3eac1e327de3355ddb49ef26b61475a65"} Aug 05 12:20:49.401017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.400979 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:49.500461 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.500427 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:49.507965 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.507920 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" podUID="2739193d-8b60-4f26-9d22-f27389fcaab8" containerName="registry" containerID="cri-o://3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8" gracePeriod=30 Aug 05 12:20:49.738063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.738039 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:20:49.845487 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845457 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9lmk\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-kube-api-access-g9lmk\") pod \"2739193d-8b60-4f26-9d22-f27389fcaab8\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " Aug 05 12:20:49.845641 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845498 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-trusted-ca\") pod \"2739193d-8b60-4f26-9d22-f27389fcaab8\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " Aug 05 12:20:49.845641 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845624 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-image-registry-private-configuration\") pod \"2739193d-8b60-4f26-9d22-f27389fcaab8\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " Aug 05 12:20:49.845747 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845669 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-bound-sa-token\") pod \"2739193d-8b60-4f26-9d22-f27389fcaab8\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " Aug 05 12:20:49.845747 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845725 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-installation-pull-secrets\") pod \"2739193d-8b60-4f26-9d22-f27389fcaab8\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " Aug 05 12:20:49.845852 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845752 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") pod \"2739193d-8b60-4f26-9d22-f27389fcaab8\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " Aug 05 12:20:49.845852 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845781 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-certificates\") pod \"2739193d-8b60-4f26-9d22-f27389fcaab8\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " Aug 05 12:20:49.845852 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845812 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2739193d-8b60-4f26-9d22-f27389fcaab8-ca-trust-extracted\") pod \"2739193d-8b60-4f26-9d22-f27389fcaab8\" (UID: \"2739193d-8b60-4f26-9d22-f27389fcaab8\") " Aug 05 12:20:49.846041 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.845894 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2739193d-8b60-4f26-9d22-f27389fcaab8" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:20:49.846179 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.846145 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-trusted-ca\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:20:49.846353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.846312 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2739193d-8b60-4f26-9d22-f27389fcaab8" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:20:49.848163 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.848134 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-kube-api-access-g9lmk" (OuterVolumeSpecName: "kube-api-access-g9lmk") pod "2739193d-8b60-4f26-9d22-f27389fcaab8" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8"). InnerVolumeSpecName "kube-api-access-g9lmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:20:49.848323 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.848296 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2739193d-8b60-4f26-9d22-f27389fcaab8" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:20:49.848458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.848432 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2739193d-8b60-4f26-9d22-f27389fcaab8" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:20:49.848562 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.848469 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2739193d-8b60-4f26-9d22-f27389fcaab8" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:20:49.848839 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.848812 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2739193d-8b60-4f26-9d22-f27389fcaab8" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:20:49.854053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.853935 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2739193d-8b60-4f26-9d22-f27389fcaab8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2739193d-8b60-4f26-9d22-f27389fcaab8" (UID: "2739193d-8b60-4f26-9d22-f27389fcaab8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:20:49.947498 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.947443 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9lmk\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-kube-api-access-g9lmk\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:20:49.947498 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.947466 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-image-registry-private-configuration\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:20:49.947498 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.947476 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-bound-sa-token\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:20:49.947498 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.947484 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2739193d-8b60-4f26-9d22-f27389fcaab8-installation-pull-secrets\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:20:49.947498 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.947492 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:20:49.947498 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.947500 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2739193d-8b60-4f26-9d22-f27389fcaab8-registry-certificates\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:20:49.947723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:49.947508 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2739193d-8b60-4f26-9d22-f27389fcaab8-ca-trust-extracted\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:20:50.118542 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.118512 2569 generic.go:356] "Generic (PLEG): container finished" podID="2739193d-8b60-4f26-9d22-f27389fcaab8" containerID="3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8" exitCode=0 Aug 05 12:20:50.118664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.118556 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" Aug 05 12:20:50.118664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.118599 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" event={"ID":"2739193d-8b60-4f26-9d22-f27389fcaab8","Type":"ContainerDied","Data":"3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8"} Aug 05 12:20:50.118664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.118645 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d86dcddf4-dgvbt" event={"ID":"2739193d-8b60-4f26-9d22-f27389fcaab8","Type":"ContainerDied","Data":"cf0c02d45c985e4ad117778cbda3b9d449851651f2e04d7707b73e7ade2f854e"} Aug 05 12:20:50.118664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.118663 2569 scope.go:117] "RemoveContainer" containerID="3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8" Aug 05 12:20:50.127292 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.127276 2569 scope.go:117] "RemoveContainer" containerID="3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8" Aug 05 12:20:50.127585 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:20:50.127565 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8\": container with ID starting with 3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8 not found: ID does not exist" containerID="3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8" Aug 05 12:20:50.127658 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.127593 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8"} err="failed to get container status \"3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8\": rpc error: code = NotFound desc = could not find container \"3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8\": container with ID starting with 3edf2213ad1d0c550dc786112a1b4ab5a33d093743cec98f3bfc4dd95dbe85b8 not found: ID does not exist" Aug 05 12:20:50.139036 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.139015 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d86dcddf4-dgvbt"] Aug 05 12:20:50.142856 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:50.142834 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5d86dcddf4-dgvbt"] Aug 05 12:20:51.441601 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:51.441559 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2739193d-8b60-4f26-9d22-f27389fcaab8" path="/var/lib/kubelet/pods/2739193d-8b60-4f26-9d22-f27389fcaab8/volumes" Aug 05 12:20:55.625086 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:20:55.624934 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:20:55.771344 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:20:55.771312 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:20:56.143928 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:56.143891 2569 generic.go:356] "Generic (PLEG): container finished" podID="1e3ef6c1-6fc9-4467-ae88-65c7b7b24436" containerID="db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a" exitCode=0 Aug 05 12:20:56.144101 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:56.143986 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" event={"ID":"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436","Type":"ContainerDied","Data":"db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a"} Aug 05 12:20:56.144285 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:56.144270 2569 scope.go:117] "RemoveContainer" containerID="db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a" Aug 05 12:20:57.051535 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:57.051486 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:57.137323 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:57.137291 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:20:57.150524 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:20:57.150491 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-s8grw" event={"ID":"1e3ef6c1-6fc9-4467-ae88-65c7b7b24436","Type":"ContainerStarted","Data":"c577f5a97d47d4b06dc3317c0bf265742276fa42a5ad5a5b84d24b545fe0285c"} Aug 05 12:21:03.471747 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:03.471713 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:21:03.475511 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:03.475485 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b665b4949-2rlhw" Aug 05 12:21:05.632836 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:05.632800 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:10.784372 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:10.784338 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:15.640702 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:15.640667 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:25.679737 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:25.679699 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:25.755242 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:25.755215 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:27.447136 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:27.447099 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:21:35.716858 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:35.716822 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:37.924756 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:37.924725 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:21:38.125795 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:38.125759 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:21:38.676188 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:38.676156 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:38.676500 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:38.676472 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:38.989368 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:38.989287 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:21:38.989732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:38.989571 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="alertmanager" containerID="cri-o://0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26" gracePeriod=120 Aug 05 12:21:38.989732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:38.989643 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy-web" containerID="cri-o://6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1" gracePeriod=120 Aug 05 12:21:38.989732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:38.989652 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy-metric" containerID="cri-o://8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50" gracePeriod=120 Aug 05 12:21:38.989732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:38.989668 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="config-reloader" containerID="cri-o://c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9" gracePeriod=120 Aug 05 12:21:38.989732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:38.989705 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy" containerID="cri-o://2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6" gracePeriod=120 Aug 05 12:21:38.990010 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:38.989760 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="prom-label-proxy" containerID="cri-o://a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b" gracePeriod=120 Aug 05 12:21:39.322185 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:39.322155 2569 generic.go:356] "Generic (PLEG): container finished" podID="d8929588-cc30-4903-80b2-26c80e4830ce" containerID="a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b" exitCode=0 Aug 05 12:21:39.322185 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:39.322183 2569 generic.go:356] "Generic (PLEG): container finished" podID="d8929588-cc30-4903-80b2-26c80e4830ce" containerID="2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6" exitCode=0 Aug 05 12:21:39.322185 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:39.322191 2569 generic.go:356] "Generic (PLEG): container finished" podID="d8929588-cc30-4903-80b2-26c80e4830ce" containerID="c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9" exitCode=0 Aug 05 12:21:39.322450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:39.322197 2569 generic.go:356] "Generic (PLEG): container finished" podID="d8929588-cc30-4903-80b2-26c80e4830ce" containerID="0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26" exitCode=0 Aug 05 12:21:39.322450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:39.322214 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerDied","Data":"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b"} Aug 05 12:21:39.322450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:39.322265 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerDied","Data":"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6"} Aug 05 12:21:39.322450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:39.322291 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerDied","Data":"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9"} Aug 05 12:21:39.322450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:39.322315 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerDied","Data":"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26"} Aug 05 12:21:40.227248 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.227225 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.329034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.328929 2569 generic.go:356] "Generic (PLEG): container finished" podID="d8929588-cc30-4903-80b2-26c80e4830ce" containerID="8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50" exitCode=0 Aug 05 12:21:40.329034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.328977 2569 generic.go:356] "Generic (PLEG): container finished" podID="d8929588-cc30-4903-80b2-26c80e4830ce" containerID="6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1" exitCode=0 Aug 05 12:21:40.329034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.328988 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerDied","Data":"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50"} Aug 05 12:21:40.329237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.329038 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.329237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.329093 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerDied","Data":"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1"} Aug 05 12:21:40.329237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.329121 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d8929588-cc30-4903-80b2-26c80e4830ce","Type":"ContainerDied","Data":"e292509f01362cc2aca9af0ee3625675f09b0267fed486cd37c88241bb3f5a15"} Aug 05 12:21:40.329237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.329154 2569 scope.go:117] "RemoveContainer" containerID="a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b" Aug 05 12:21:40.335102 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335082 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-main-tls\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335111 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-web\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335135 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-cluster-tls-config\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335169 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-web-config\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335215 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-config-out\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335246 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335299 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335327 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8czrd\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-kube-api-access-8czrd\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335352 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-tls-assets\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335377 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-config-volume\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.335450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335416 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.336278 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335793 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-metrics-client-ca\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.336278 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335809 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:21:40.336278 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.335830 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-main-db\") pod \"d8929588-cc30-4903-80b2-26c80e4830ce\" (UID: \"d8929588-cc30-4903-80b2-26c80e4830ce\") " Aug 05 12:21:40.336278 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.336165 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.336604 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.336448 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:21:40.336604 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.336576 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:21:40.338108 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.338032 2569 scope.go:117] "RemoveContainer" containerID="8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50" Aug 05 12:21:40.338514 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.338302 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:21:40.339340 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.338648 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:21:40.339340 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.338985 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:21:40.339340 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.339087 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-kube-api-access-8czrd" (OuterVolumeSpecName: "kube-api-access-8czrd") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "kube-api-access-8czrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:21:40.339340 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.339160 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:21:40.339340 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.339301 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:21:40.339722 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.339697 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-config-volume" (OuterVolumeSpecName: "config-volume") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:21:40.340284 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.340258 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-config-out" (OuterVolumeSpecName: "config-out") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:21:40.343757 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.343739 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:21:40.350754 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.350724 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-web-config" (OuterVolumeSpecName: "web-config") pod "d8929588-cc30-4903-80b2-26c80e4830ce" (UID: "d8929588-cc30-4903-80b2-26c80e4830ce"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:21:40.351310 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.351294 2569 scope.go:117] "RemoveContainer" containerID="2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6" Aug 05 12:21:40.357506 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.357492 2569 scope.go:117] "RemoveContainer" containerID="6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1" Aug 05 12:21:40.363844 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.363829 2569 scope.go:117] "RemoveContainer" containerID="c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9" Aug 05 12:21:40.370122 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.370106 2569 scope.go:117] "RemoveContainer" containerID="0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26" Aug 05 12:21:40.376651 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.376637 2569 scope.go:117] "RemoveContainer" containerID="24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b" Aug 05 12:21:40.383019 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.383000 2569 scope.go:117] "RemoveContainer" containerID="a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b" Aug 05 12:21:40.383290 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:40.383269 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b\": container with ID starting with a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b not found: ID does not exist" containerID="a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b" Aug 05 12:21:40.383335 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.383299 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b"} err="failed to get container status \"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b\": rpc error: code = NotFound desc = could not find container \"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b\": container with ID starting with a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b not found: ID does not exist" Aug 05 12:21:40.383335 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.383318 2569 scope.go:117] "RemoveContainer" containerID="8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50" Aug 05 12:21:40.383553 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:40.383534 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50\": container with ID starting with 8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50 not found: ID does not exist" containerID="8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50" Aug 05 12:21:40.383603 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.383558 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50"} err="failed to get container status \"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50\": rpc error: code = NotFound desc = could not find container \"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50\": container with ID starting with 8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50 not found: ID does not exist" Aug 05 12:21:40.383603 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.383574 2569 scope.go:117] "RemoveContainer" containerID="2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6" Aug 05 12:21:40.383790 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:40.383772 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6\": container with ID starting with 2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6 not found: ID does not exist" containerID="2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6" Aug 05 12:21:40.383829 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.383804 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6"} err="failed to get container status \"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6\": rpc error: code = NotFound desc = could not find container \"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6\": container with ID starting with 2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6 not found: ID does not exist" Aug 05 12:21:40.383829 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.383819 2569 scope.go:117] "RemoveContainer" containerID="6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1" Aug 05 12:21:40.384068 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:40.384051 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1\": container with ID starting with 6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1 not found: ID does not exist" containerID="6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1" Aug 05 12:21:40.384170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.384070 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1"} err="failed to get container status \"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1\": rpc error: code = NotFound desc = could not find container \"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1\": container with ID starting with 6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1 not found: ID does not exist" Aug 05 12:21:40.384170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.384083 2569 scope.go:117] "RemoveContainer" containerID="c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9" Aug 05 12:21:40.384324 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:40.384308 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9\": container with ID starting with c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9 not found: ID does not exist" containerID="c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9" Aug 05 12:21:40.384364 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.384327 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9"} err="failed to get container status \"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9\": rpc error: code = NotFound desc = could not find container \"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9\": container with ID starting with c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9 not found: ID does not exist" Aug 05 12:21:40.384364 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.384347 2569 scope.go:117] "RemoveContainer" containerID="0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26" Aug 05 12:21:40.384577 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:40.384561 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26\": container with ID starting with 0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26 not found: ID does not exist" containerID="0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26" Aug 05 12:21:40.384631 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.384579 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26"} err="failed to get container status \"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26\": rpc error: code = NotFound desc = could not find container \"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26\": container with ID starting with 0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26 not found: ID does not exist" Aug 05 12:21:40.384631 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.384592 2569 scope.go:117] "RemoveContainer" containerID="24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b" Aug 05 12:21:40.384783 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:40.384769 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b\": container with ID starting with 24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b not found: ID does not exist" containerID="24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b" Aug 05 12:21:40.384821 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.384785 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b"} err="failed to get container status \"24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b\": rpc error: code = NotFound desc = could not find container \"24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b\": container with ID starting with 24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b not found: ID does not exist" Aug 05 12:21:40.384821 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.384796 2569 scope.go:117] "RemoveContainer" containerID="a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b" Aug 05 12:21:40.385026 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385010 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b"} err="failed to get container status \"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b\": rpc error: code = NotFound desc = could not find container \"a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b\": container with ID starting with a127c8f20f7ab9dcca87e3aef9578c22b041a2792ba252febb7287f5cf8ae30b not found: ID does not exist" Aug 05 12:21:40.385026 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385025 2569 scope.go:117] "RemoveContainer" containerID="8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50" Aug 05 12:21:40.385226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385212 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50"} err="failed to get container status \"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50\": rpc error: code = NotFound desc = could not find container \"8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50\": container with ID starting with 8db3713c2ceeda24d4a15824b12793ee27146daea7556d605bd76c812bffae50 not found: ID does not exist" Aug 05 12:21:40.385271 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385226 2569 scope.go:117] "RemoveContainer" containerID="2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6" Aug 05 12:21:40.385434 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385415 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6"} err="failed to get container status \"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6\": rpc error: code = NotFound desc = could not find container \"2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6\": container with ID starting with 2bce3787079bf2588439c70b17345dd8e60a71882e27e83805427e0b60f60cc6 not found: ID does not exist" Aug 05 12:21:40.385484 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385435 2569 scope.go:117] "RemoveContainer" containerID="6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1" Aug 05 12:21:40.385641 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385627 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1"} err="failed to get container status \"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1\": rpc error: code = NotFound desc = could not find container \"6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1\": container with ID starting with 6beb7ad8c13b44c4f579366c8dcb3df4403c4de556b2a8a079db5058eb30bbe1 not found: ID does not exist" Aug 05 12:21:40.385681 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385641 2569 scope.go:117] "RemoveContainer" containerID="c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9" Aug 05 12:21:40.385854 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385835 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9"} err="failed to get container status \"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9\": rpc error: code = NotFound desc = could not find container \"c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9\": container with ID starting with c3dcc4b4e11813fbc141ae90da0d6945438ae187a280e88169ed5969f4d7e5d9 not found: ID does not exist" Aug 05 12:21:40.385894 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.385855 2569 scope.go:117] "RemoveContainer" containerID="0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26" Aug 05 12:21:40.386082 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.386067 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26"} err="failed to get container status \"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26\": rpc error: code = NotFound desc = could not find container \"0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26\": container with ID starting with 0d324b067ecfded69e28ccdd5ac58e10227476f17aa625b19b65653fb65bdc26 not found: ID does not exist" Aug 05 12:21:40.386125 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.386081 2569 scope.go:117] "RemoveContainer" containerID="24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b" Aug 05 12:21:40.386295 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.386278 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b"} err="failed to get container status \"24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b\": rpc error: code = NotFound desc = could not find container \"24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b\": container with ID starting with 24da4e1a24b727682e404b600610277571dccff4960570505737631c096b8e1b not found: ID does not exist" Aug 05 12:21:40.437135 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437108 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8czrd\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-kube-api-access-8czrd\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437135 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437133 2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d8929588-cc30-4903-80b2-26c80e4830ce-tls-assets\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437142 2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-config-volume\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437151 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437160 2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d8929588-cc30-4903-80b2-26c80e4830ce-metrics-client-ca\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437168 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-alertmanager-main-db\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437177 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-main-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437186 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437195 2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-cluster-tls-config\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437204 2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-web-config\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437212 2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d8929588-cc30-4903-80b2-26c80e4830ce-config-out\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.437261 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.437220 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d8929588-cc30-4903-80b2-26c80e4830ce-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:21:40.652901 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.652824 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:21:40.656128 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.656099 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:21:40.681366 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681338 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:21:40.681604 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681592 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="prom-label-proxy" Aug 05 12:21:40.681646 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681606 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="prom-label-proxy" Aug 05 12:21:40.681646 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681614 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy" Aug 05 12:21:40.681646 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681620 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy" Aug 05 12:21:40.681646 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681630 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="alertmanager" Aug 05 12:21:40.681646 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681635 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="alertmanager" Aug 05 12:21:40.681646 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681643 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy-metric" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681648 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy-metric" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681659 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="config-reloader" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681664 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="config-reloader" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681672 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="init-config-reloader" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681677 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="init-config-reloader" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681684 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy-web" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681690 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy-web" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681698 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2739193d-8b60-4f26-9d22-f27389fcaab8" containerName="registry" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681702 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2739193d-8b60-4f26-9d22-f27389fcaab8" containerName="registry" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681744 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="alertmanager" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681751 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy-web" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681760 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681766 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="kube-rbac-proxy-metric" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681772 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="prom-label-proxy" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681778 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" containerName="config-reloader" Aug 05 12:21:40.681815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.681784 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2739193d-8b60-4f26-9d22-f27389fcaab8" containerName="registry" Aug 05 12:21:40.686551 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.686535 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.688815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.688797 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 05 12:21:40.688911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.688800 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 05 12:21:40.689122 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.689104 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 05 12:21:40.689195 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.689172 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 05 12:21:40.689195 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.689178 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 05 12:21:40.689366 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.689350 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 05 12:21:40.689413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.689400 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 05 12:21:40.689632 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.689618 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 05 12:21:40.689673 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.689638 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zhkcg\"" Aug 05 12:21:40.695207 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.695184 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 05 12:21:40.700317 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.700297 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:21:40.739930 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.739905 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e567e0c8-24d3-483a-93b0-7e2233c9f693-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.739941 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e567e0c8-24d3-483a-93b0-7e2233c9f693-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.739989 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2vn2\" (UniqueName: \"kubernetes.io/projected/e567e0c8-24d3-483a-93b0-7e2233c9f693-kube-api-access-l2vn2\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740019 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e567e0c8-24d3-483a-93b0-7e2233c9f693-config-out\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740047 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740072 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e567e0c8-24d3-483a-93b0-7e2233c9f693-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740111 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-config-volume\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740181 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740205 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-web-config\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740233 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740441 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740321 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740441 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740349 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.740441 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.740369 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e567e0c8-24d3-483a-93b0-7e2233c9f693-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.793253 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:40.793229 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:40.841202 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841174 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841208 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841225 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e567e0c8-24d3-483a-93b0-7e2233c9f693-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841267 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e567e0c8-24d3-483a-93b0-7e2233c9f693-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841290 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e567e0c8-24d3-483a-93b0-7e2233c9f693-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841312 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vn2\" (UniqueName: \"kubernetes.io/projected/e567e0c8-24d3-483a-93b0-7e2233c9f693-kube-api-access-l2vn2\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841339 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e567e0c8-24d3-483a-93b0-7e2233c9f693-config-out\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841555 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841428 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841555 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841469 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e567e0c8-24d3-483a-93b0-7e2233c9f693-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841555 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841511 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-config-volume\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841735 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841554 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841735 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841581 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-web-config\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841735 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841612 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.841909 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.841734 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e567e0c8-24d3-483a-93b0-7e2233c9f693-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.842220 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.842161 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e567e0c8-24d3-483a-93b0-7e2233c9f693-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.844378 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.844154 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e567e0c8-24d3-483a-93b0-7e2233c9f693-config-out\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.844378 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.844257 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e567e0c8-24d3-483a-93b0-7e2233c9f693-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.844378 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.844325 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.844630 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.844456 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.844630 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.844480 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.844630 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.844553 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.844630 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.844587 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-config-volume\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.844836 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.844806 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-web-config\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.845115 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.845095 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e567e0c8-24d3-483a-93b0-7e2233c9f693-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.846111 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.846092 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e567e0c8-24d3-483a-93b0-7e2233c9f693-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.849342 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.849322 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2vn2\" (UniqueName: \"kubernetes.io/projected/e567e0c8-24d3-483a-93b0-7e2233c9f693-kube-api-access-l2vn2\") pod \"alertmanager-main-0\" (UID: \"e567e0c8-24d3-483a-93b0-7e2233c9f693\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:40.996581 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:40.996509 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 05 12:21:41.128877 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:41.128848 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 12:21:41.131789 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:21:41.131761 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode567e0c8_24d3_483a_93b0_7e2233c9f693.slice/crio-7e075c5f662bb0b255769a6462b1882c297b923483321ca34b7648baec7a11a7 WatchSource:0}: Error finding container 7e075c5f662bb0b255769a6462b1882c297b923483321ca34b7648baec7a11a7: Status 404 returned error can't find the container with id 7e075c5f662bb0b255769a6462b1882c297b923483321ca34b7648baec7a11a7 Aug 05 12:21:41.335110 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:41.335073 2569 generic.go:356] "Generic (PLEG): container finished" podID="e567e0c8-24d3-483a-93b0-7e2233c9f693" containerID="aa7248d9b503536018eb0595150cc2ab78046aba1aa8b64dc5156234acf9a407" exitCode=0 Aug 05 12:21:41.335444 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:41.335130 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e567e0c8-24d3-483a-93b0-7e2233c9f693","Type":"ContainerDied","Data":"aa7248d9b503536018eb0595150cc2ab78046aba1aa8b64dc5156234acf9a407"} Aug 05 12:21:41.335444 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:41.335184 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e567e0c8-24d3-483a-93b0-7e2233c9f693","Type":"ContainerStarted","Data":"7e075c5f662bb0b255769a6462b1882c297b923483321ca34b7648baec7a11a7"} Aug 05 12:21:41.443032 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:41.443001 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8929588-cc30-4903-80b2-26c80e4830ce" path="/var/lib/kubelet/pods/d8929588-cc30-4903-80b2-26c80e4830ce/volumes" Aug 05 12:21:42.342449 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:42.342380 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e567e0c8-24d3-483a-93b0-7e2233c9f693","Type":"ContainerStarted","Data":"1e4b82f5fa0d21e258554d542bb4e32c7244e7b05d542b00dcbc432b0c544f11"} Aug 05 12:21:42.342449 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:42.342444 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e567e0c8-24d3-483a-93b0-7e2233c9f693","Type":"ContainerStarted","Data":"26f76a5c873bf985ae74e9e181c488f9655d6c6b83d9b442566a7c71ab6a2167"} Aug 05 12:21:42.342927 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:42.342476 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e567e0c8-24d3-483a-93b0-7e2233c9f693","Type":"ContainerStarted","Data":"201d95ca61148a4f70e49e3f55da90764b9548eac946e9aff360be86b2bce7d8"} Aug 05 12:21:42.342927 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:42.342500 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e567e0c8-24d3-483a-93b0-7e2233c9f693","Type":"ContainerStarted","Data":"2d037ac2e34a8b6ad765f6f555d46b88e66d5e76e168535581422bced80d3de1"} Aug 05 12:21:42.342927 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:42.342524 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e567e0c8-24d3-483a-93b0-7e2233c9f693","Type":"ContainerStarted","Data":"303d8e2b1ac7ba9e9f8c62d1daf5d86215e70c11aa53540cd1a83e26e1502ae8"} Aug 05 12:21:42.342927 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:42.342549 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e567e0c8-24d3-483a-93b0-7e2233c9f693","Type":"ContainerStarted","Data":"45e6e262e3fd5c4eb5ffb5d3bd92504ed0146431e6ec866d111b13c65a03890b"} Aug 05 12:21:42.370371 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:42.370324 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.355260086 podStartE2EDuration="2.37030447s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.044384ms" isStatefulPod=false podCreationTimestamp="2026-08-05 12:21:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:21:42.368854229 +0000 UTC m=+167.434446146" watchObservedRunningTime="2026-08-05 12:21:42.37030447 +0000 UTC m=+167.435896369" Aug 05 12:21:43.006631 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.006595 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn"] Aug 05 12:21:43.010423 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.010395 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.012657 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.012624 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Aug 05 12:21:43.012820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.012625 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Aug 05 12:21:43.012820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.012633 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Aug 05 12:21:43.012962 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.012858 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4blbv\"" Aug 05 12:21:43.012962 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.012929 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Aug 05 12:21:43.013393 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.013375 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Aug 05 12:21:43.017707 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.017558 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Aug 05 12:21:43.026300 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.026278 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn"] Aug 05 12:21:43.059353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.059330 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-telemeter-client-tls\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.059467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.059368 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.059467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.059446 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-serving-certs-ca-bundle\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.059541 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.059468 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7xcc\" (UniqueName: \"kubernetes.io/projected/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-kube-api-access-b7xcc\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.059541 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.059503 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.059541 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.059531 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-secret-telemeter-client\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.059626 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.059554 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-metrics-client-ca\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.059659 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.059618 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-federate-client-tls\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.160543 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.160503 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-serving-certs-ca-bundle\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.160543 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.160548 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7xcc\" (UniqueName: \"kubernetes.io/projected/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-kube-api-access-b7xcc\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.160696 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.160654 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.160728 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.160692 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-secret-telemeter-client\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.160728 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.160710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-metrics-client-ca\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.160858 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.160841 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-federate-client-tls\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.160906 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.160882 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-telemeter-client-tls\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.161120 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.161097 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.161320 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.161287 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-serving-certs-ca-bundle\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.161450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.161420 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-metrics-client-ca\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.161596 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.161574 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.163228 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.163208 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-telemeter-client-tls\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.163336 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.163316 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-federate-client-tls\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.163520 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.163504 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.163638 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.163619 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-secret-telemeter-client\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.168556 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.168532 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7xcc\" (UniqueName: \"kubernetes.io/projected/8b4c9d0c-8874-443a-b6a9-f34d4b7d8131-kube-api-access-b7xcc\") pod \"telemeter-client-5d4cbd77b7-7t9tn\" (UID: \"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131\") " pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.320283 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.320214 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" Aug 05 12:21:43.450543 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:43.450392 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn"] Aug 05 12:21:43.452904 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:21:43.452870 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4c9d0c_8874_443a_b6a9_f34d4b7d8131.slice/crio-1a2a7681e2eb01445d3bc5bd4fff85247d86b21f9da1a2994ba792f4711d2470 WatchSource:0}: Error finding container 1a2a7681e2eb01445d3bc5bd4fff85247d86b21f9da1a2994ba792f4711d2470: Status 404 returned error can't find the container with id 1a2a7681e2eb01445d3bc5bd4fff85247d86b21f9da1a2994ba792f4711d2470 Aug 05 12:21:44.352350 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:44.352313 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" event={"ID":"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131","Type":"ContainerStarted","Data":"1a2a7681e2eb01445d3bc5bd4fff85247d86b21f9da1a2994ba792f4711d2470"} Aug 05 12:21:45.357660 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:45.357612 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" event={"ID":"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131","Type":"ContainerStarted","Data":"f088a2eaef129e04308b8f10145a707382d5eed38190f6502e2a9bee884ce311"} Aug 05 12:21:45.727591 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:21:45.727518 2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef6c1_6fc9_4467_ae88_65c7b7b24436.slice/crio-db4e884ab06edea39b827d453f807a43bb92819dacb940882db37075d0010c0a.scope\": RecentStats: unable to find data in memory cache]" Aug 05 12:21:46.364259 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.364225 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" event={"ID":"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131","Type":"ContainerStarted","Data":"83970f4eb0c92a0fdad45fce7edb9ec5f0bf286fa02a6bdd7d03de730e8e01dd"} Aug 05 12:21:46.364630 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.364283 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" event={"ID":"8b4c9d0c-8874-443a-b6a9-f34d4b7d8131","Type":"ContainerStarted","Data":"e03f98f557c7e35379d595cba5d0838f6d17f114ec07770ea7a1387c1ce93b0a"} Aug 05 12:21:46.388195 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.388147 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5d4cbd77b7-7t9tn" podStartSLOduration=2.549943021 podStartE2EDuration="4.388132674s" totalImagesPullingTime="1.838189653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:21:46.386982521 +0000 UTC m=+171.452574419" watchObservedRunningTime="2026-08-05 12:21:46.388132674 +0000 UTC m=+171.453724571" Aug 05 12:21:46.475736 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.475699 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-648dd898ff-lllh9"] Aug 05 12:21:46.479151 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.479128 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.488770 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.488744 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648dd898ff-lllh9"] Aug 05 12:21:46.529804 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.529775 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:21:46.592099 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.592068 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-serving-cert\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.592219 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.592103 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-config\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.592219 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.592129 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-oauth-config\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.592219 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.592208 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smwh8\" (UniqueName: \"kubernetes.io/projected/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-kube-api-access-smwh8\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.592351 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.592258 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-trusted-ca-bundle\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.592351 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.592291 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-service-ca\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.592351 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.592339 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-oauth-serving-cert\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.593182 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.593164 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:21:46.693206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.693154 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-serving-cert\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.693206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.693179 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-config\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.693206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.693201 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-oauth-config\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.693421 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.693220 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smwh8\" (UniqueName: \"kubernetes.io/projected/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-kube-api-access-smwh8\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.693421 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.693253 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-trusted-ca-bundle\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.693421 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.693310 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-service-ca\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.693421 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.693364 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-oauth-serving-cert\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.693980 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.693945 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-config\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.694179 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.694157 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-trusted-ca-bundle\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.694227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.694191 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-oauth-serving-cert\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.694491 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.694470 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-service-ca\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.695548 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.695525 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-serving-cert\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.695634 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.695622 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-oauth-config\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.701514 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.701497 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smwh8\" (UniqueName: \"kubernetes.io/projected/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-kube-api-access-smwh8\") pod \"console-648dd898ff-lllh9\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.789377 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.789353 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:46.917001 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:46.916946 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648dd898ff-lllh9"] Aug 05 12:21:46.919185 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:21:46.919159 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f1b7b58_11a0_48f7_bf38_8ef3b65f27a0.slice/crio-deaad26e30337e7cdd635356c37190051729094b9d180b4e96459674b0ef4f80 WatchSource:0}: Error finding container deaad26e30337e7cdd635356c37190051729094b9d180b4e96459674b0ef4f80: Status 404 returned error can't find the container with id deaad26e30337e7cdd635356c37190051729094b9d180b4e96459674b0ef4f80 Aug 05 12:21:47.370096 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:47.370058 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648dd898ff-lllh9" event={"ID":"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0","Type":"ContainerStarted","Data":"e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1"} Aug 05 12:21:47.370450 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:47.370117 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648dd898ff-lllh9" event={"ID":"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0","Type":"ContainerStarted","Data":"deaad26e30337e7cdd635356c37190051729094b9d180b4e96459674b0ef4f80"} Aug 05 12:21:47.394129 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:47.394068 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-648dd898ff-lllh9" podStartSLOduration=1.394053264 podStartE2EDuration="1.394053264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:21:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:21:47.391683769 +0000 UTC m=+172.457275668" watchObservedRunningTime="2026-08-05 12:21:47.394053264 +0000 UTC m=+172.459645162" Aug 05 12:21:56.790464 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:56.790419 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:56.790980 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:56.790486 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:56.795363 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:56.795342 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:57.414900 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:57.414874 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:21:57.465287 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:57.465250 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7548c6b4c-jsblx"] Aug 05 12:21:57.575814 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:57.575783 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:21:57.628737 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:57.628705 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:21:57.691312 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:21:57.691233 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:22:17.163338 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:17.163298 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:22:17.245285 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:17.245244 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:22:22.488191 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.488148 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7548c6b4c-jsblx" podUID="3a4ec894-3eb2-4564-8208-03a277fad951" containerName="console" containerID="cri-o://2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d" gracePeriod=15 Aug 05 12:22:22.729235 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.729212 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:22:22.793501 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.793470 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8kwb\" (UniqueName: \"kubernetes.io/projected/3a4ec894-3eb2-4564-8208-03a277fad951-kube-api-access-w8kwb\") pod \"3a4ec894-3eb2-4564-8208-03a277fad951\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " Aug 05 12:22:22.793666 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.793514 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-oauth-config\") pod \"3a4ec894-3eb2-4564-8208-03a277fad951\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " Aug 05 12:22:22.793666 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.793536 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-service-ca\") pod \"3a4ec894-3eb2-4564-8208-03a277fad951\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " Aug 05 12:22:22.793666 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.793569 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-trusted-ca-bundle\") pod \"3a4ec894-3eb2-4564-8208-03a277fad951\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " Aug 05 12:22:22.793666 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.793654 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-serving-cert\") pod \"3a4ec894-3eb2-4564-8208-03a277fad951\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " Aug 05 12:22:22.793895 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.793699 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-console-config\") pod \"3a4ec894-3eb2-4564-8208-03a277fad951\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " Aug 05 12:22:22.793895 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.793733 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-oauth-serving-cert\") pod \"3a4ec894-3eb2-4564-8208-03a277fad951\" (UID: \"3a4ec894-3eb2-4564-8208-03a277fad951\") " Aug 05 12:22:22.794040 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.794004 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3a4ec894-3eb2-4564-8208-03a277fad951" (UID: "3a4ec894-3eb2-4564-8208-03a277fad951"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:22:22.794100 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.793923 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-service-ca" (OuterVolumeSpecName: "service-ca") pod "3a4ec894-3eb2-4564-8208-03a277fad951" (UID: "3a4ec894-3eb2-4564-8208-03a277fad951"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:22:22.794100 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.794069 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-console-config" (OuterVolumeSpecName: "console-config") pod "3a4ec894-3eb2-4564-8208-03a277fad951" (UID: "3a4ec894-3eb2-4564-8208-03a277fad951"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:22:22.794208 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.794129 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3a4ec894-3eb2-4564-8208-03a277fad951" (UID: "3a4ec894-3eb2-4564-8208-03a277fad951"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:22:22.795811 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.795788 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a4ec894-3eb2-4564-8208-03a277fad951-kube-api-access-w8kwb" (OuterVolumeSpecName: "kube-api-access-w8kwb") pod "3a4ec894-3eb2-4564-8208-03a277fad951" (UID: "3a4ec894-3eb2-4564-8208-03a277fad951"). InnerVolumeSpecName "kube-api-access-w8kwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:22:22.795811 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.795802 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3a4ec894-3eb2-4564-8208-03a277fad951" (UID: "3a4ec894-3eb2-4564-8208-03a277fad951"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:22:22.795927 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.795862 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3a4ec894-3eb2-4564-8208-03a277fad951" (UID: "3a4ec894-3eb2-4564-8208-03a277fad951"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:22:22.895079 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.895049 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-serving-cert\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:22:22.895079 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.895075 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-console-config\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:22:22.895226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.895089 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-oauth-serving-cert\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:22:22.895226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.895104 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8kwb\" (UniqueName: \"kubernetes.io/projected/3a4ec894-3eb2-4564-8208-03a277fad951-kube-api-access-w8kwb\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:22:22.895226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.895116 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a4ec894-3eb2-4564-8208-03a277fad951-console-oauth-config\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:22:22.895226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.895128 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-service-ca\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:22:22.895226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:22.895141 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a4ec894-3eb2-4564-8208-03a277fad951-trusted-ca-bundle\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:22:23.512515 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.512483 2569 generic.go:356] "Generic (PLEG): container finished" podID="3a4ec894-3eb2-4564-8208-03a277fad951" containerID="2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d" exitCode=0 Aug 05 12:22:23.512913 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.512523 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7548c6b4c-jsblx" Aug 05 12:22:23.512913 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.512552 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7548c6b4c-jsblx" event={"ID":"3a4ec894-3eb2-4564-8208-03a277fad951","Type":"ContainerDied","Data":"2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d"} Aug 05 12:22:23.512913 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.512600 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7548c6b4c-jsblx" event={"ID":"3a4ec894-3eb2-4564-8208-03a277fad951","Type":"ContainerDied","Data":"f1c3d257f22da63b57e5567c67f1c09afbc65c8f19db128e78ac66f07dfa141f"} Aug 05 12:22:23.512913 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.512633 2569 scope.go:117] "RemoveContainer" containerID="2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d" Aug 05 12:22:23.520741 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.520725 2569 scope.go:117] "RemoveContainer" containerID="2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d" Aug 05 12:22:23.521012 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:22:23.520992 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d\": container with ID starting with 2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d not found: ID does not exist" containerID="2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d" Aug 05 12:22:23.521076 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.521019 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d"} err="failed to get container status \"2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d\": rpc error: code = NotFound desc = could not find container \"2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d\": container with ID starting with 2068edb60c49639108abec64818153423940aa5548987a84c63d2980a839682d not found: ID does not exist" Aug 05 12:22:23.527429 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.527408 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7548c6b4c-jsblx"] Aug 05 12:22:23.532422 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:23.532405 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7548c6b4c-jsblx"] Aug 05 12:22:25.441008 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:25.440974 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a4ec894-3eb2-4564-8208-03a277fad951" path="/var/lib/kubelet/pods/3a4ec894-3eb2-4564-8208-03a277fad951/volumes" Aug 05 12:22:51.797373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:51.797340 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:22:53.192902 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:53.192870 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:22:53.272009 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:53.271977 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:22:55.788198 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:22:55.788168 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:01.854758 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:01.854712 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cddbd4669-8zgg5"] Aug 05 12:23:01.855309 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:01.855062 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a4ec894-3eb2-4564-8208-03a277fad951" containerName="console" Aug 05 12:23:01.855309 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:01.855074 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a4ec894-3eb2-4564-8208-03a277fad951" containerName="console" Aug 05 12:23:01.855309 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:01.855133 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a4ec894-3eb2-4564-8208-03a277fad951" containerName="console" Aug 05 12:23:01.859752 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:01.859716 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:01.874229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:01.874202 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cddbd4669-8zgg5"] Aug 05 12:23:01.914538 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:01.914509 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:02.000537 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.000505 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-oauth-config\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.000537 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.000535 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-trusted-ca-bundle\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.000718 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.000574 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-config\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.000718 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.000617 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-service-ca\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.000718 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.000641 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-oauth-serving-cert\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.000718 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.000666 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq94h\" (UniqueName: \"kubernetes.io/projected/6b03bcb0-79b4-4448-b886-7cf846ce07f7-kube-api-access-lq94h\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.000718 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.000685 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-serving-cert\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.018765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.018733 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:02.102019 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.101984 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-oauth-serving-cert\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102019 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102023 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq94h\" (UniqueName: \"kubernetes.io/projected/6b03bcb0-79b4-4448-b886-7cf846ce07f7-kube-api-access-lq94h\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102230 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102046 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-serving-cert\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102230 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102070 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-oauth-config\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102230 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102086 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-trusted-ca-bundle\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102230 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102126 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-config\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102230 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102180 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-service-ca\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102867 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102839 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-oauth-serving-cert\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102996 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102839 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-config\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.102996 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.102910 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-service-ca\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.103135 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.103110 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-trusted-ca-bundle\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.104614 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.104595 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-serving-cert\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.104736 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.104656 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-oauth-config\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.110322 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.110276 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq94h\" (UniqueName: \"kubernetes.io/projected/6b03bcb0-79b4-4448-b886-7cf846ce07f7-kube-api-access-lq94h\") pod \"console-cddbd4669-8zgg5\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.170154 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.170114 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:02.297292 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.297265 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cddbd4669-8zgg5"] Aug 05 12:23:02.299965 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:23:02.299921 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b03bcb0_79b4_4448_b886_7cf846ce07f7.slice/crio-fd8899f5aceb52dd5d6a51509ba3b473cfdba11491d4373ce87a8907eceac701 WatchSource:0}: Error finding container fd8899f5aceb52dd5d6a51509ba3b473cfdba11491d4373ce87a8907eceac701: Status 404 returned error can't find the container with id fd8899f5aceb52dd5d6a51509ba3b473cfdba11491d4373ce87a8907eceac701 Aug 05 12:23:02.669912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.669878 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cddbd4669-8zgg5" event={"ID":"6b03bcb0-79b4-4448-b886-7cf846ce07f7","Type":"ContainerStarted","Data":"aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118"} Aug 05 12:23:02.670092 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.669933 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cddbd4669-8zgg5" event={"ID":"6b03bcb0-79b4-4448-b886-7cf846ce07f7","Type":"ContainerStarted","Data":"fd8899f5aceb52dd5d6a51509ba3b473cfdba11491d4373ce87a8907eceac701"} Aug 05 12:23:02.688240 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:02.688190 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cddbd4669-8zgg5" podStartSLOduration=1.688174007 podStartE2EDuration="1.688174007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:23:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:23:02.686942108 +0000 UTC m=+247.752534001" watchObservedRunningTime="2026-08-05 12:23:02.688174007 +0000 UTC m=+247.753765977" Aug 05 12:23:05.264834 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:05.264803 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:05.365918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:05.365884 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:11.709390 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:11.709354 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:11.810096 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:11.810049 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:11.925366 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:11.925337 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:12.170766 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:12.170726 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:12.170935 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:12.170785 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:12.180642 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:12.180615 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:12.711515 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:12.711489 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:23:12.758232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:12.758200 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648dd898ff-lllh9"] Aug 05 12:23:12.863385 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:12.863351 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:12.965127 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:12.965052 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:13.018848 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:13.018820 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:23:37.780201 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:37.780161 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-648dd898ff-lllh9" podUID="4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" containerName="console" containerID="cri-o://e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1" gracePeriod=15 Aug 05 12:23:38.016733 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.016709 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:23:38.073092 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073030 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smwh8\" (UniqueName: \"kubernetes.io/projected/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-kube-api-access-smwh8\") pod \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " Aug 05 12:23:38.073092 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073064 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-serving-cert\") pod \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " Aug 05 12:23:38.073255 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073091 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-trusted-ca-bundle\") pod \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " Aug 05 12:23:38.073255 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073137 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-service-ca\") pod \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " Aug 05 12:23:38.073255 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073179 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-config\") pod \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " Aug 05 12:23:38.073255 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073211 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-oauth-serving-cert\") pod \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " Aug 05 12:23:38.073457 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073262 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-oauth-config\") pod \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\" (UID: \"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0\") " Aug 05 12:23:38.073529 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073500 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" (UID: "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:23:38.073642 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073599 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-service-ca" (OuterVolumeSpecName: "service-ca") pod "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" (UID: "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:23:38.073765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073696 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" (UID: "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:23:38.073765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.073719 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-config" (OuterVolumeSpecName: "console-config") pod "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" (UID: "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:23:38.075225 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.075202 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-kube-api-access-smwh8" (OuterVolumeSpecName: "kube-api-access-smwh8") pod "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" (UID: "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0"). InnerVolumeSpecName "kube-api-access-smwh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:23:38.075328 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.075230 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" (UID: "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:23:38.075328 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.075261 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" (UID: "4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:23:38.174709 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.174686 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smwh8\" (UniqueName: \"kubernetes.io/projected/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-kube-api-access-smwh8\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:23:38.174709 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.174708 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-serving-cert\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:23:38.174845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.174718 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-trusted-ca-bundle\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:23:38.174845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.174728 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-service-ca\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:23:38.174845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.174741 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-config\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:23:38.174845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.174753 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-oauth-serving-cert\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:23:38.174845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.174763 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0-console-oauth-config\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:23:38.817500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.817468 2569 generic.go:356] "Generic (PLEG): container finished" podID="4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" containerID="e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1" exitCode=0 Aug 05 12:23:38.817881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.817512 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648dd898ff-lllh9" Aug 05 12:23:38.817881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.817554 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648dd898ff-lllh9" event={"ID":"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0","Type":"ContainerDied","Data":"e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1"} Aug 05 12:23:38.817881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.817607 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648dd898ff-lllh9" event={"ID":"4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0","Type":"ContainerDied","Data":"deaad26e30337e7cdd635356c37190051729094b9d180b4e96459674b0ef4f80"} Aug 05 12:23:38.817881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.817633 2569 scope.go:117] "RemoveContainer" containerID="e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1" Aug 05 12:23:38.826943 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.826927 2569 scope.go:117] "RemoveContainer" containerID="e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1" Aug 05 12:23:38.827210 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:23:38.827193 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1\": container with ID starting with e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1 not found: ID does not exist" containerID="e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1" Aug 05 12:23:38.827262 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.827217 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1"} err="failed to get container status \"e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1\": rpc error: code = NotFound desc = could not find container \"e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1\": container with ID starting with e044754a701b52fbf16f6ec1d4ae01b5c27f075ddedd1b55b0577c78ba64a2f1 not found: ID does not exist" Aug 05 12:23:38.838324 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.838301 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648dd898ff-lllh9"] Aug 05 12:23:38.841415 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:38.841395 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-648dd898ff-lllh9"] Aug 05 12:23:39.442212 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:39.442181 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" path="/var/lib/kubelet/pods/4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0/volumes" Aug 05 12:23:55.294388 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:55.294363 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:23:55.297730 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:55.297705 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:23:55.306225 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:23:55.306203 2569 kubelet.go:1648] "Image garbage collection succeeded" Aug 05 12:25:27.563181 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:27.563140 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:25:29.097297 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:29.097265 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:25:33.945036 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:33.944991 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:25:34.042872 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.042843 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:25:34.587356 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.587322 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:25:34.593509 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.593480 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj"] Aug 05 12:25:34.593868 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.593851 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" containerName="console" Aug 05 12:25:34.593939 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.593871 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" containerName="console" Aug 05 12:25:34.594012 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.593992 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f1b7b58-11a0-48f7-bf38-8ef3b65f27a0" containerName="console" Aug 05 12:25:34.596824 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.596805 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:25:34.599031 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.599012 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Aug 05 12:25:34.599031 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.599022 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-qztbp\"" Aug 05 12:25:34.599618 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.599602 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Aug 05 12:25:34.613407 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.613390 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj"] Aug 05 12:25:34.655226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.655200 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x4g5\" (UniqueName: \"kubernetes.io/projected/482edd92-5ce9-4017-b649-999f37de1131-kube-api-access-7x4g5\") pod \"feast-operator-controller-manager-544888d49-fqcjj\" (UID: \"482edd92-5ce9-4017-b649-999f37de1131\") " pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:25:34.755782 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.755759 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x4g5\" (UniqueName: \"kubernetes.io/projected/482edd92-5ce9-4017-b649-999f37de1131-kube-api-access-7x4g5\") pod \"feast-operator-controller-manager-544888d49-fqcjj\" (UID: \"482edd92-5ce9-4017-b649-999f37de1131\") " pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:25:34.763311 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.763287 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x4g5\" (UniqueName: \"kubernetes.io/projected/482edd92-5ce9-4017-b649-999f37de1131-kube-api-access-7x4g5\") pod \"feast-operator-controller-manager-544888d49-fqcjj\" (UID: \"482edd92-5ce9-4017-b649-999f37de1131\") " pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:25:34.906808 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:34.906743 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:25:35.028609 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:35.028587 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj"] Aug 05 12:25:35.030723 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:25:35.030695 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482edd92_5ce9_4017_b649_999f37de1131.slice/crio-1f69e547e279725f54f4a322119a3ea52d85266827e13f049412207511e67917 WatchSource:0}: Error finding container 1f69e547e279725f54f4a322119a3ea52d85266827e13f049412207511e67917: Status 404 returned error can't find the container with id 1f69e547e279725f54f4a322119a3ea52d85266827e13f049412207511e67917 Aug 05 12:25:35.032552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:35.032534 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 12:25:35.270042 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:35.270014 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" event={"ID":"482edd92-5ce9-4017-b649-999f37de1131","Type":"ContainerStarted","Data":"1f69e547e279725f54f4a322119a3ea52d85266827e13f049412207511e67917"} Aug 05 12:25:38.284498 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:38.284457 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" event={"ID":"482edd92-5ce9-4017-b649-999f37de1131","Type":"ContainerStarted","Data":"f68e5e190c8c6853f301b73f654968b146d3e1d3e24dec434acae57453b7e341"} Aug 05 12:25:38.284877 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:38.284531 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:25:38.299840 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:38.299796 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" podStartSLOduration=1.9402293670000001 podStartE2EDuration="4.299779924s" totalImagesPullingTime="2.359550557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:25:38.298784151 +0000 UTC m=+403.364376050" watchObservedRunningTime="2026-08-05 12:25:38.299779924 +0000 UTC m=+403.365371823" Aug 05 12:25:49.290210 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:49.290180 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:25:58.403311 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:58.403276 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:25:58.453963 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:25:58.453932 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:05.568181 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.568145 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dc6dc6d44-f5c6z"] Aug 05 12:26:05.571667 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.571643 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.584351 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.584321 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc6dc6d44-f5c6z"] Aug 05 12:26:05.617565 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.617535 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:05.713458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.713433 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-oauth-serving-cert\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.713594 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.713466 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-trusted-ca-bundle\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.713594 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.713483 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-config\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.713594 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.713501 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-serving-cert\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.713594 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.713580 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-service-ca\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.713803 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.713611 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs82c\" (UniqueName: \"kubernetes.io/projected/812f98d1-0dc9-4f99-b6f9-28322a514c81-kube-api-access-rs82c\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.713803 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.713686 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-oauth-config\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.814721 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.814691 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-service-ca\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.814721 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.814723 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs82c\" (UniqueName: \"kubernetes.io/projected/812f98d1-0dc9-4f99-b6f9-28322a514c81-kube-api-access-rs82c\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.814911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.814755 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-oauth-config\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.814911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.814799 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-oauth-serving-cert\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.814911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.814817 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-trusted-ca-bundle\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.814911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.814833 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-config\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.814911 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.814852 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-serving-cert\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.815553 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.815529 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-service-ca\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.815635 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.815618 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-oauth-serving-cert\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.815717 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.815694 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-config\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.815757 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.815733 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/812f98d1-0dc9-4f99-b6f9-28322a514c81-trusted-ca-bundle\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.817522 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.817501 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-serving-cert\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.817609 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.817499 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/812f98d1-0dc9-4f99-b6f9-28322a514c81-console-oauth-config\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.822420 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.822379 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs82c\" (UniqueName: \"kubernetes.io/projected/812f98d1-0dc9-4f99-b6f9-28322a514c81-kube-api-access-rs82c\") pod \"console-5dc6dc6d44-f5c6z\" (UID: \"812f98d1-0dc9-4f99-b6f9-28322a514c81\") " pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:05.882485 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:05.882467 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:06.011045 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:06.008236 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc6dc6d44-f5c6z"] Aug 05 12:26:06.014863 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:26:06.014832 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812f98d1_0dc9_4f99_b6f9_28322a514c81.slice/crio-eb0efda82b0e869653e84b471848f7569b8b3713653f075431ded7acd8521be8 WatchSource:0}: Error finding container eb0efda82b0e869653e84b471848f7569b8b3713653f075431ded7acd8521be8: Status 404 returned error can't find the container with id eb0efda82b0e869653e84b471848f7569b8b3713653f075431ded7acd8521be8 Aug 05 12:26:06.403699 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:06.403662 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc6dc6d44-f5c6z" event={"ID":"812f98d1-0dc9-4f99-b6f9-28322a514c81","Type":"ContainerStarted","Data":"97bf493242e74357032d8e2872a6754fe027beda3e9672d4aa56d243307825ca"} Aug 05 12:26:06.403850 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:06.403719 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc6dc6d44-f5c6z" event={"ID":"812f98d1-0dc9-4f99-b6f9-28322a514c81","Type":"ContainerStarted","Data":"eb0efda82b0e869653e84b471848f7569b8b3713653f075431ded7acd8521be8"} Aug 05 12:26:06.420619 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:06.420573 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dc6dc6d44-f5c6z" podStartSLOduration=1.420559017 podStartE2EDuration="1.420559017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:26:06.41989298 +0000 UTC m=+431.485484879" watchObservedRunningTime="2026-08-05 12:26:06.420559017 +0000 UTC m=+431.486150914" Aug 05 12:26:13.918299 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:13.918265 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:14.018121 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.018089 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:14.174699 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.174622 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:14.247658 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.247626 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:14.252256 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.252232 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl"] Aug 05 12:26:14.259371 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.259349 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.261757 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.261728 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Aug 05 12:26:14.262066 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.262044 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Aug 05 12:26:14.262172 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.262096 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Aug 05 12:26:14.262172 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.262115 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Aug 05 12:26:14.262172 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.262124 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Aug 05 12:26:14.262333 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.262177 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hqjg6\"" Aug 05 12:26:14.262333 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.262115 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Aug 05 12:26:14.262333 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.262311 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Aug 05 12:26:14.268451 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.268429 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl"] Aug 05 12:26:14.361395 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.361371 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:14.386492 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.386470 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-registry-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.386591 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.386522 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rscwv\" (UniqueName: \"kubernetes.io/projected/2652dfb0-580a-4e60-b333-487c33666b60-kube-api-access-rscwv\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.386591 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.386577 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.386661 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.386629 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.386661 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.386646 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-ui-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.386730 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.386662 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2652dfb0-580a-4e60-b333-487c33666b60-feast-data\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.459514 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.459453 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:14.487451 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.487427 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.487538 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.487452 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-ui-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.487538 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.487488 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2652dfb0-580a-4e60-b333-487c33666b60-feast-data\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.487623 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.487545 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-registry-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.487623 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:26:14.487568 2569 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Aug 05 12:26:14.487623 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.487613 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rscwv\" (UniqueName: \"kubernetes.io/projected/2652dfb0-580a-4e60-b333-487c33666b60-kube-api-access-rscwv\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.487623 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:26:14.487622 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls podName:2652dfb0-580a-4e60-b333-487c33666b60 nodeName:}" failed. No retries permitted until 2026-08-05 12:26:14.987608592 +0000 UTC m=+440.053200468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls") pod "feast-simple-feast-setup-6b89b8d8fb-5pvbl" (UID: "2652dfb0-580a-4e60-b333-487c33666b60") : secret "feast-simple-feast-setup-online-tls" not found Aug 05 12:26:14.487789 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.487662 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.487789 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:26:14.487755 2569 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Aug 05 12:26:14.487868 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:26:14.487791 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls podName:2652dfb0-580a-4e60-b333-487c33666b60 nodeName:}" failed. No retries permitted until 2026-08-05 12:26:14.987779364 +0000 UTC m=+440.053371240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls") pod "feast-simple-feast-setup-6b89b8d8fb-5pvbl" (UID: "2652dfb0-580a-4e60-b333-487c33666b60") : secret "feast-simple-feast-setup-offline-tls" not found Aug 05 12:26:14.488013 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.487991 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2652dfb0-580a-4e60-b333-487c33666b60-feast-data\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.490230 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.490202 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-ui-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.490314 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.490202 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-registry-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.497654 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.497633 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rscwv\" (UniqueName: \"kubernetes.io/projected/2652dfb0-580a-4e60-b333-487c33666b60-kube-api-access-rscwv\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.558786 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.558751 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:14.730136 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.730053 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:14.992011 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.991922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.992011 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.991993 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.994342 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.994318 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:14.994452 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:14.994377 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-5pvbl\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:15.178341 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:15.178312 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:15.311137 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:15.310984 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl"] Aug 05 12:26:15.313825 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:26:15.313801 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2652dfb0_580a_4e60_b333_487c33666b60.slice/crio-b70ad7f5436f6da95ddeaf94bb3c72dd106f2ffde8a115798187c12ca21024dd WatchSource:0}: Error finding container b70ad7f5436f6da95ddeaf94bb3c72dd106f2ffde8a115798187c12ca21024dd: Status 404 returned error can't find the container with id b70ad7f5436f6da95ddeaf94bb3c72dd106f2ffde8a115798187c12ca21024dd Aug 05 12:26:15.447291 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:15.447260 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerStarted","Data":"b70ad7f5436f6da95ddeaf94bb3c72dd106f2ffde8a115798187c12ca21024dd"} Aug 05 12:26:15.883341 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:15.883304 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:15.883522 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:15.883370 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:15.888046 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:15.888019 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:16.460361 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:16.460330 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dc6dc6d44-f5c6z" Aug 05 12:26:16.511655 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:16.508739 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cddbd4669-8zgg5"] Aug 05 12:26:16.618825 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:16.618756 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:16.967299 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:16.967256 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:26:40.575577 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:40.575541 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerStarted","Data":"f3003c350c25c9f7aedb2b4b8c5f3a0b266c3b216bc852252b359a9a9c8951bb"} Aug 05 12:26:41.537551 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.537498 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cddbd4669-8zgg5" podUID="6b03bcb0-79b4-4448-b886-7cf846ce07f7" containerName="console" containerID="cri-o://aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118" gracePeriod=15 Aug 05 12:26:41.770026 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.769999 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:26:41.834967 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.834880 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-oauth-serving-cert\") pod \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " Aug 05 12:26:41.834967 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.834910 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lq94h\" (UniqueName: \"kubernetes.io/projected/6b03bcb0-79b4-4448-b886-7cf846ce07f7-kube-api-access-lq94h\") pod \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " Aug 05 12:26:41.834967 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.834940 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-trusted-ca-bundle\") pod \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " Aug 05 12:26:41.835206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.834993 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-config\") pod \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " Aug 05 12:26:41.835206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.835045 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-oauth-config\") pod \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " Aug 05 12:26:41.835206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.835168 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-service-ca\") pod \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " Aug 05 12:26:41.835206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.835202 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-serving-cert\") pod \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\" (UID: \"6b03bcb0-79b4-4448-b886-7cf846ce07f7\") " Aug 05 12:26:41.835418 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.835285 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6b03bcb0-79b4-4448-b886-7cf846ce07f7" (UID: "6b03bcb0-79b4-4448-b886-7cf846ce07f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:26:41.835492 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.835469 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-config" (OuterVolumeSpecName: "console-config") pod "6b03bcb0-79b4-4448-b886-7cf846ce07f7" (UID: "6b03bcb0-79b4-4448-b886-7cf846ce07f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:26:41.835552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.835505 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-oauth-serving-cert\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:26:41.835552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.835512 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6b03bcb0-79b4-4448-b886-7cf846ce07f7" (UID: "6b03bcb0-79b4-4448-b886-7cf846ce07f7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:26:41.835819 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.835792 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "6b03bcb0-79b4-4448-b886-7cf846ce07f7" (UID: "6b03bcb0-79b4-4448-b886-7cf846ce07f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:26:41.837275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.837239 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6b03bcb0-79b4-4448-b886-7cf846ce07f7" (UID: "6b03bcb0-79b4-4448-b886-7cf846ce07f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:26:41.837369 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.837268 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6b03bcb0-79b4-4448-b886-7cf846ce07f7" (UID: "6b03bcb0-79b4-4448-b886-7cf846ce07f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:26:41.837563 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.837542 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b03bcb0-79b4-4448-b886-7cf846ce07f7-kube-api-access-lq94h" (OuterVolumeSpecName: "kube-api-access-lq94h") pod "6b03bcb0-79b4-4448-b886-7cf846ce07f7" (UID: "6b03bcb0-79b4-4448-b886-7cf846ce07f7"). InnerVolumeSpecName "kube-api-access-lq94h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:26:41.936518 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.936492 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-service-ca\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:26:41.936518 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.936516 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-serving-cert\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:26:41.936663 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.936531 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq94h\" (UniqueName: \"kubernetes.io/projected/6b03bcb0-79b4-4448-b886-7cf846ce07f7-kube-api-access-lq94h\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:26:41.936663 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.936544 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-trusted-ca-bundle\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:26:41.936663 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.936556 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-config\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:26:41.936663 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:41.936568 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b03bcb0-79b4-4448-b886-7cf846ce07f7-console-oauth-config\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:26:42.585150 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.585113 2569 generic.go:356] "Generic (PLEG): container finished" podID="6b03bcb0-79b4-4448-b886-7cf846ce07f7" containerID="aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118" exitCode=0 Aug 05 12:26:42.585337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.585182 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cddbd4669-8zgg5" Aug 05 12:26:42.585337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.585201 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cddbd4669-8zgg5" event={"ID":"6b03bcb0-79b4-4448-b886-7cf846ce07f7","Type":"ContainerDied","Data":"aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118"} Aug 05 12:26:42.585337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.585262 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cddbd4669-8zgg5" event={"ID":"6b03bcb0-79b4-4448-b886-7cf846ce07f7","Type":"ContainerDied","Data":"fd8899f5aceb52dd5d6a51509ba3b473cfdba11491d4373ce87a8907eceac701"} Aug 05 12:26:42.585337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.585289 2569 scope.go:117] "RemoveContainer" containerID="aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118" Aug 05 12:26:42.593871 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.593852 2569 scope.go:117] "RemoveContainer" containerID="aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118" Aug 05 12:26:42.594132 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:26:42.594114 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118\": container with ID starting with aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118 not found: ID does not exist" containerID="aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118" Aug 05 12:26:42.594189 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.594140 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118"} err="failed to get container status \"aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118\": rpc error: code = NotFound desc = could not find container \"aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118\": container with ID starting with aea636cd4fac559f061e143087a4483ff2e040d5a9ae938350ee215c9040e118 not found: ID does not exist" Aug 05 12:26:42.607487 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.607466 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cddbd4669-8zgg5"] Aug 05 12:26:42.611555 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:42.611535 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cddbd4669-8zgg5"] Aug 05 12:26:43.442003 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:43.441973 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b03bcb0-79b4-4448-b886-7cf846ce07f7" path="/var/lib/kubelet/pods/6b03bcb0-79b4-4448-b886-7cf846ce07f7/volumes" Aug 05 12:26:44.595808 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:44.595780 2569 generic.go:356] "Generic (PLEG): container finished" podID="2652dfb0-580a-4e60-b333-487c33666b60" containerID="f3003c350c25c9f7aedb2b4b8c5f3a0b266c3b216bc852252b359a9a9c8951bb" exitCode=0 Aug 05 12:26:44.596212 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:44.595860 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerDied","Data":"f3003c350c25c9f7aedb2b4b8c5f3a0b266c3b216bc852252b359a9a9c8951bb"} Aug 05 12:26:45.601927 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:45.601892 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerStarted","Data":"514c99e496405fe0a5c73ca3203a159f17d5ceca64502e36c402ad34b6a950ff"} Aug 05 12:26:50.625190 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:50.625161 2569 generic.go:356] "Generic (PLEG): container finished" podID="2652dfb0-580a-4e60-b333-487c33666b60" containerID="514c99e496405fe0a5c73ca3203a159f17d5ceca64502e36c402ad34b6a950ff" exitCode=0 Aug 05 12:26:50.625574 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:50.625238 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerDied","Data":"514c99e496405fe0a5c73ca3203a159f17d5ceca64502e36c402ad34b6a950ff"} Aug 05 12:26:51.636757 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:51.636711 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerStarted","Data":"1da6dbd715688a624396f19feecba3e0f21c566811fdc6cf3ee64f598f67e7b0"} Aug 05 12:26:51.637350 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:51.636781 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerStarted","Data":"7ec0953e9380db92f642ff031a880a54b23f6c07efb5b981287e0f7d26426fce"} Aug 05 12:26:51.637350 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:51.636804 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerStarted","Data":"c4b3faaf069edf750df803e4fd1ffc2a58df2af7a26087406843d788824964bd"} Aug 05 12:26:51.637350 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:51.636833 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerStarted","Data":"7f230aad195ba6b3142040119e20cdc3cb99129a10d9be3e37b115c77e683b82"} Aug 05 12:26:51.664456 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:51.664395 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podStartSLOduration=13.276569709 podStartE2EDuration="37.664374787s" totalImagesPullingTime="24.387805078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:26:51.664161475 +0000 UTC m=+476.729753386" watchObservedRunningTime="2026-08-05 12:26:51.664374787 +0000 UTC m=+476.729966687" Aug 05 12:26:54.179379 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:54.179143 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:54.179379 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:54.179210 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:54.179379 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:54.179231 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:54.179996 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:54.179634 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:54.181745 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:54.181705 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Aug 05 12:26:54.181881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:54.181755 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Aug 05 12:26:54.181881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:54.181760 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Aug 05 12:26:54.181881 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:54.181871 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Aug 05 12:26:55.179235 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:55.179196 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:55.179511 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:55.179486 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:55.179511 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:55.179518 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:55.180044 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:55.179537 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:57.180071 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:57.180038 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:57.180532 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:57.180191 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:57.180597 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:57.180582 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:57.181095 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:57.181070 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:57.181232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:57.181139 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:57.181232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:57.181174 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:57.190437 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:57.190416 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:57.674185 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:57.674149 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:26:58.114859 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:26:58.114820 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:27:59.709113 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:27:59.709021 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:27:59.733317 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:27:59.733277 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl"] Aug 05 12:27:59.733609 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:27:59.733556 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="registry" containerID="cri-o://7f230aad195ba6b3142040119e20cdc3cb99129a10d9be3e37b115c77e683b82" gracePeriod=30 Aug 05 12:27:59.733748 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:27:59.733627 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="offline" containerID="cri-o://7ec0953e9380db92f642ff031a880a54b23f6c07efb5b981287e0f7d26426fce" gracePeriod=30 Aug 05 12:27:59.733748 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:27:59.733666 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="online" containerID="cri-o://c4b3faaf069edf750df803e4fd1ffc2a58df2af7a26087406843d788824964bd" gracePeriod=30 Aug 05 12:27:59.733748 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:27:59.733637 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="ui" containerID="cri-o://1da6dbd715688a624396f19feecba3e0f21c566811fdc6cf3ee64f598f67e7b0" gracePeriod=30 Aug 05 12:27:59.806807 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:27:59.806769 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:27:59.919791 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:27:59.919738 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:28:00.022918 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.022872 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:28:00.075818 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.075779 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27"] Aug 05 12:28:00.076253 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.076227 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b03bcb0-79b4-4448-b886-7cf846ce07f7" containerName="console" Aug 05 12:28:00.076253 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.076249 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b03bcb0-79b4-4448-b886-7cf846ce07f7" containerName="console" Aug 05 12:28:00.076454 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.076325 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b03bcb0-79b4-4448-b886-7cf846ce07f7" containerName="console" Aug 05 12:28:00.078917 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.078897 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.081782 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.081761 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-xk5kz\"" Aug 05 12:28:00.096353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.096331 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27"] Aug 05 12:28:00.171428 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.171397 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-online-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.171751 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.171449 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb8s8\" (UniqueName: \"kubernetes.io/projected/76e37654-2f33-4479-911c-2616f6aadc09-kube-api-access-jb8s8\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.171751 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.171510 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-registry-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.171751 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.171578 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-offline-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.171751 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.171663 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/76e37654-2f33-4479-911c-2616f6aadc09-feast-data\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.171751 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.171740 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-ui-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.206047 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.206009 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:28:00.273005 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.272927 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-offline-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.273005 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.272992 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/76e37654-2f33-4479-911c-2616f6aadc09-feast-data\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.273227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.273026 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-ui-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.273227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.273049 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-online-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.273227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.273069 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb8s8\" (UniqueName: \"kubernetes.io/projected/76e37654-2f33-4479-911c-2616f6aadc09-kube-api-access-jb8s8\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.273227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.273110 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-registry-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.273453 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.273434 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/76e37654-2f33-4479-911c-2616f6aadc09-feast-data\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.275373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.275350 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-offline-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.275486 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.275469 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-online-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.275580 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.275556 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-registry-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.275831 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.275815 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-ui-tls\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.283723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.283656 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb8s8\" (UniqueName: \"kubernetes.io/projected/76e37654-2f33-4479-911c-2616f6aadc09-kube-api-access-jb8s8\") pod \"feast-simple-feast-setup-6b89b8d8fb-nxr27\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.302040 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.302006 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:28:00.389015 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.388988 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:00.399771 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.399737 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:28:00.470081 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.470036 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:28:00.527156 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.527128 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27"] Aug 05 12:28:00.528574 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:28:00.528547 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76e37654_2f33_4479_911c_2616f6aadc09.slice/crio-8075afe3357ce83a75579a67f400620c6b12498dab1002fc04a5aa3479c2d970 WatchSource:0}: Error finding container 8075afe3357ce83a75579a67f400620c6b12498dab1002fc04a5aa3479c2d970: Status 404 returned error can't find the container with id 8075afe3357ce83a75579a67f400620c6b12498dab1002fc04a5aa3479c2d970 Aug 05 12:28:00.931875 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.931790 2569 generic.go:356] "Generic (PLEG): container finished" podID="2652dfb0-580a-4e60-b333-487c33666b60" containerID="1da6dbd715688a624396f19feecba3e0f21c566811fdc6cf3ee64f598f67e7b0" exitCode=0 Aug 05 12:28:00.931875 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.931814 2569 generic.go:356] "Generic (PLEG): container finished" podID="2652dfb0-580a-4e60-b333-487c33666b60" containerID="c4b3faaf069edf750df803e4fd1ffc2a58df2af7a26087406843d788824964bd" exitCode=0 Aug 05 12:28:00.932327 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.931867 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerDied","Data":"1da6dbd715688a624396f19feecba3e0f21c566811fdc6cf3ee64f598f67e7b0"} Aug 05 12:28:00.932327 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.931919 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerDied","Data":"c4b3faaf069edf750df803e4fd1ffc2a58df2af7a26087406843d788824964bd"} Aug 05 12:28:00.933338 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.933315 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerStarted","Data":"1ab490708e791cf3137f6381a4cc7c0e99b21d932a16f75458a4e6b81d5a520f"} Aug 05 12:28:00.933469 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:00.933352 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerStarted","Data":"8075afe3357ce83a75579a67f400620c6b12498dab1002fc04a5aa3479c2d970"} Aug 05 12:28:05.954720 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:05.954685 2569 generic.go:356] "Generic (PLEG): container finished" podID="76e37654-2f33-4479-911c-2616f6aadc09" containerID="1ab490708e791cf3137f6381a4cc7c0e99b21d932a16f75458a4e6b81d5a520f" exitCode=0 Aug 05 12:28:05.955140 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:05.954758 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerDied","Data":"1ab490708e791cf3137f6381a4cc7c0e99b21d932a16f75458a4e6b81d5a520f"} Aug 05 12:28:06.961388 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:06.961353 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerStarted","Data":"db405fbf93e37e3f1906ef4e3802b1bc0ce708b180ee6e5ae581eadbe9ced469"} Aug 05 12:28:07.181512 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:07.181466 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Aug 05 12:28:07.672407 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:07.672366 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Aug 05 12:28:11.989871 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:11.989834 2569 generic.go:356] "Generic (PLEG): container finished" podID="76e37654-2f33-4479-911c-2616f6aadc09" containerID="db405fbf93e37e3f1906ef4e3802b1bc0ce708b180ee6e5ae581eadbe9ced469" exitCode=0 Aug 05 12:28:11.990359 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:11.989867 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerDied","Data":"db405fbf93e37e3f1906ef4e3802b1bc0ce708b180ee6e5ae581eadbe9ced469"} Aug 05 12:28:13.000695 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:13.000649 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerStarted","Data":"8435f201d1312532c6665c3cf219a5c3fb329e3504848d23445b0c54b52dff53"} Aug 05 12:28:13.001206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:13.000710 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerStarted","Data":"fb87c79886c30d78d8fee5be53ab2a0fbebdeddd62fac90868c0da0e885cf97a"} Aug 05 12:28:13.001206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:13.000731 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerStarted","Data":"9a366d0dc5aa4be573d7e1a959690ac9348268d152a1ae23c9138fb7579599a7"} Aug 05 12:28:13.001206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:13.000748 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerStarted","Data":"a3c79cab4ef9d7c7fb736fd4d0dfdbda387fe015299f0c5ec55de7181c23db0b"} Aug 05 12:28:15.389435 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:15.389396 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:15.389926 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:15.389458 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:15.389926 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:15.389478 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:15.389926 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:15.389498 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:15.391714 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:15.391680 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Aug 05 12:28:15.391969 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:15.391703 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused" Aug 05 12:28:15.392077 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:15.391760 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Aug 05 12:28:15.392077 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:15.391808 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused" Aug 05 12:28:17.181783 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:17.181734 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Aug 05 12:28:17.671482 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:17.671437 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Aug 05 12:28:18.391243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.391210 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.391628 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.391464 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.391628 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.391589 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.391893 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.391869 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.392008 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.391917 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.392008 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.391936 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.392136 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.392099 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.392408 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.392387 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.392474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.392444 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.396180 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.396165 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:18.433385 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:18.433338 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podStartSLOduration=18.433322469 podStartE2EDuration="18.433322469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:28:13.050455694 +0000 UTC m=+558.116047593" watchObservedRunningTime="2026-08-05 12:28:18.433322469 +0000 UTC m=+563.498914371" Aug 05 12:28:19.037107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:19.037072 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:19.040356 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:19.040335 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:28:19.602933 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:19.602896 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:28:27.180996 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:27.180911 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Aug 05 12:28:27.181397 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:27.181088 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:28:27.407250 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:27.407212 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:28:27.671617 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:27.671582 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Aug 05 12:28:27.671811 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:27.671713 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:28:30.084210 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.084177 2569 generic.go:356] "Generic (PLEG): container finished" podID="2652dfb0-580a-4e60-b333-487c33666b60" containerID="7ec0953e9380db92f642ff031a880a54b23f6c07efb5b981287e0f7d26426fce" exitCode=137 Aug 05 12:28:30.084210 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.084201 2569 generic.go:356] "Generic (PLEG): container finished" podID="2652dfb0-580a-4e60-b333-487c33666b60" containerID="7f230aad195ba6b3142040119e20cdc3cb99129a10d9be3e37b115c77e683b82" exitCode=137 Aug 05 12:28:30.084640 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.084216 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerDied","Data":"7ec0953e9380db92f642ff031a880a54b23f6c07efb5b981287e0f7d26426fce"} Aug 05 12:28:30.084640 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.084257 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerDied","Data":"7f230aad195ba6b3142040119e20cdc3cb99129a10d9be3e37b115c77e683b82"} Aug 05 12:28:30.378315 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.378292 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:28:30.428200 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.428174 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2652dfb0-580a-4e60-b333-487c33666b60-feast-data\") pod \"2652dfb0-580a-4e60-b333-487c33666b60\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " Aug 05 12:28:30.428358 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.428209 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rscwv\" (UniqueName: \"kubernetes.io/projected/2652dfb0-580a-4e60-b333-487c33666b60-kube-api-access-rscwv\") pod \"2652dfb0-580a-4e60-b333-487c33666b60\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " Aug 05 12:28:30.428358 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.428237 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls\") pod \"2652dfb0-580a-4e60-b333-487c33666b60\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " Aug 05 12:28:30.428358 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.428269 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls\") pod \"2652dfb0-580a-4e60-b333-487c33666b60\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " Aug 05 12:28:30.428537 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.428376 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-registry-tls\") pod \"2652dfb0-580a-4e60-b333-487c33666b60\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " Aug 05 12:28:30.428537 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.428403 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-ui-tls\") pod \"2652dfb0-580a-4e60-b333-487c33666b60\" (UID: \"2652dfb0-580a-4e60-b333-487c33666b60\") " Aug 05 12:28:30.428865 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.428798 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2652dfb0-580a-4e60-b333-487c33666b60-feast-data" (OuterVolumeSpecName: "feast-data") pod "2652dfb0-580a-4e60-b333-487c33666b60" (UID: "2652dfb0-580a-4e60-b333-487c33666b60"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:28:30.430668 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.430629 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls" (OuterVolumeSpecName: "online-tls") pod "2652dfb0-580a-4e60-b333-487c33666b60" (UID: "2652dfb0-580a-4e60-b333-487c33666b60"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:28:30.430776 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.430721 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2652dfb0-580a-4e60-b333-487c33666b60-kube-api-access-rscwv" (OuterVolumeSpecName: "kube-api-access-rscwv") pod "2652dfb0-580a-4e60-b333-487c33666b60" (UID: "2652dfb0-580a-4e60-b333-487c33666b60"). InnerVolumeSpecName "kube-api-access-rscwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:28:30.430856 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.430811 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2652dfb0-580a-4e60-b333-487c33666b60" (UID: "2652dfb0-580a-4e60-b333-487c33666b60"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:28:30.430856 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.430828 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2652dfb0-580a-4e60-b333-487c33666b60" (UID: "2652dfb0-580a-4e60-b333-487c33666b60"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:28:30.430925 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.430884 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2652dfb0-580a-4e60-b333-487c33666b60" (UID: "2652dfb0-580a-4e60-b333-487c33666b60"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:28:30.529039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.529006 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2652dfb0-580a-4e60-b333-487c33666b60-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:28:30.529039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.529038 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rscwv\" (UniqueName: \"kubernetes.io/projected/2652dfb0-580a-4e60-b333-487c33666b60-kube-api-access-rscwv\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:28:30.529039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.529047 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:28:30.529217 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.529056 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:28:30.529217 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.529064 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:28:30.529217 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:30.529071 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2652dfb0-580a-4e60-b333-487c33666b60-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:28:31.091104 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.091078 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" Aug 05 12:28:31.091505 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.091072 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl" event={"ID":"2652dfb0-580a-4e60-b333-487c33666b60","Type":"ContainerDied","Data":"b70ad7f5436f6da95ddeaf94bb3c72dd106f2ffde8a115798187c12ca21024dd"} Aug 05 12:28:31.091505 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.091201 2569 scope.go:117] "RemoveContainer" containerID="1da6dbd715688a624396f19feecba3e0f21c566811fdc6cf3ee64f598f67e7b0" Aug 05 12:28:31.100257 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.100239 2569 scope.go:117] "RemoveContainer" containerID="7ec0953e9380db92f642ff031a880a54b23f6c07efb5b981287e0f7d26426fce" Aug 05 12:28:31.108216 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.108197 2569 scope.go:117] "RemoveContainer" containerID="c4b3faaf069edf750df803e4fd1ffc2a58df2af7a26087406843d788824964bd" Aug 05 12:28:31.115409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.115393 2569 scope.go:117] "RemoveContainer" containerID="7f230aad195ba6b3142040119e20cdc3cb99129a10d9be3e37b115c77e683b82" Aug 05 12:28:31.121254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.121232 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl"] Aug 05 12:28:31.123275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.123259 2569 scope.go:117] "RemoveContainer" containerID="514c99e496405fe0a5c73ca3203a159f17d5ceca64502e36c402ad34b6a950ff" Aug 05 12:28:31.130152 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.130138 2569 scope.go:117] "RemoveContainer" containerID="f3003c350c25c9f7aedb2b4b8c5f3a0b266c3b216bc852252b359a9a9c8951bb" Aug 05 12:28:31.133130 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.133110 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-5pvbl"] Aug 05 12:28:31.442310 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:31.442233 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2652dfb0-580a-4e60-b333-487c33666b60" path="/var/lib/kubelet/pods/2652dfb0-580a-4e60-b333-487c33666b60/volumes" Aug 05 12:28:55.324427 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:55.324397 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:28:55.325065 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:28:55.324770 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:29:06.119049 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:06.119006 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:29:21.432547 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.432458 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:29:21.530379 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.530345 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:29:21.631852 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.631821 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:29:21.705151 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705068 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:29:21.705424 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705403 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr"] Aug 05 12:29:21.705785 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705769 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="online" Aug 05 12:29:21.705827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705787 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="online" Aug 05 12:29:21.705827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705798 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="ui" Aug 05 12:29:21.705827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705803 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="ui" Aug 05 12:29:21.705827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705817 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="offline" Aug 05 12:29:21.705827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705822 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="offline" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705832 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="feast-init" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705841 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="feast-init" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705854 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="feast-apply" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705860 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="feast-apply" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705867 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="registry" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705872 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="registry" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705964 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="registry" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705974 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="ui" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705982 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="online" Aug 05 12:29:21.706075 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.705989 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2652dfb0-580a-4e60-b333-487c33666b60" containerName="offline" Aug 05 12:29:21.709306 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.709290 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.711473 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.711446 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Aug 05 12:29:21.711619 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.711535 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Aug 05 12:29:21.711619 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.711556 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Aug 05 12:29:21.711787 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.711657 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Aug 05 12:29:21.711878 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.711858 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Aug 05 12:29:21.711979 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.711942 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-z997n\"" Aug 05 12:29:21.712030 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.711973 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Aug 05 12:29:21.712178 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.712166 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Aug 05 12:29:21.720474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.720454 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr"] Aug 05 12:29:21.818108 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.818067 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:29:21.821151 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.821123 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-online-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.821288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.821154 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv6l6\" (UniqueName: \"kubernetes.io/projected/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-kube-api-access-jv6l6\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.821288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.821185 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-feast-data\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.821288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.821219 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-registry-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.821288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.821239 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-offline-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.821444 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.821307 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.920162 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.920135 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:29:21.921923 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.921902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-registry-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.922025 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.921934 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-offline-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.922025 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.921990 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.922116 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.922036 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-online-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.922116 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.922088 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv6l6\" (UniqueName: \"kubernetes.io/projected/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-kube-api-access-jv6l6\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.922202 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.922128 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-feast-data\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.922202 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:29:21.922134 2569 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Aug 05 12:29:21.922276 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:29:21.922261 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls podName:f69aa275-59ee-4a37-8a8e-1f0daf56cbd9 nodeName:}" failed. No retries permitted until 2026-08-05 12:29:22.422241044 +0000 UTC m=+627.487832921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls") pod "feast-simple-feast-remote-setup-6dccd445d5-6mtxr" (UID: "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9") : secret "feast-simple-feast-remote-setup-ui-tls" not found Aug 05 12:29:21.922492 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.922467 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-feast-data\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.922627 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.922610 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-registry-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.924403 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.924385 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-offline-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.924534 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.924517 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-online-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:21.930627 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:21.930600 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv6l6\" (UniqueName: \"kubernetes.io/projected/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-kube-api-access-jv6l6\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:22.237965 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:22.237929 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:29:22.427081 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:22.427048 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:22.429409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:22.429392 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls\") pod \"feast-simple-feast-remote-setup-6dccd445d5-6mtxr\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:22.618925 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:22.618899 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:22.748565 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:22.748541 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr"] Aug 05 12:29:22.750850 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:29:22.750825 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf69aa275_59ee_4a37_8a8e_1f0daf56cbd9.slice/crio-e719f49ce58129a0152380982cf949eeb185e95ec4cabe516794cfc5d9a48644 WatchSource:0}: Error finding container e719f49ce58129a0152380982cf949eeb185e95ec4cabe516794cfc5d9a48644: Status 404 returned error can't find the container with id e719f49ce58129a0152380982cf949eeb185e95ec4cabe516794cfc5d9a48644 Aug 05 12:29:23.303007 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:23.302970 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerStarted","Data":"0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5"} Aug 05 12:29:23.303007 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:23.303016 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerStarted","Data":"e719f49ce58129a0152380982cf949eeb185e95ec4cabe516794cfc5d9a48644"} Aug 05 12:29:28.328622 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:28.328584 2569 generic.go:356] "Generic (PLEG): container finished" podID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerID="0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5" exitCode=0 Aug 05 12:29:28.329081 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:28.328653 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerDied","Data":"0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5"} Aug 05 12:29:29.335181 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:29.335145 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerStarted","Data":"d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9"} Aug 05 12:29:34.360777 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:34.360688 2569 generic.go:356] "Generic (PLEG): container finished" podID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerID="d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9" exitCode=0 Aug 05 12:29:34.360777 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:34.360724 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerDied","Data":"d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9"} Aug 05 12:29:35.369721 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:35.369681 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerStarted","Data":"3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9"} Aug 05 12:29:35.370231 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:35.369744 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerStarted","Data":"70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4"} Aug 05 12:29:35.370231 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:35.369766 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerStarted","Data":"e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd"} Aug 05 12:29:35.395684 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:35.395613 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podStartSLOduration=14.395591265 podStartE2EDuration="14.395591265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:29:35.393783915 +0000 UTC m=+640.459375813" watchObservedRunningTime="2026-08-05 12:29:35.395591265 +0000 UTC m=+640.461183166" Aug 05 12:29:37.620041 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:37.620006 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:37.620518 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:37.620133 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:37.620518 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:37.620156 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:37.622145 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:37.622113 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Aug 05 12:29:37.622145 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:37.622133 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Aug 05 12:29:37.622307 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:37.622132 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.32:8816: connect: connection refused" Aug 05 12:29:40.620809 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:40.620780 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:40.621217 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:40.620927 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:40.621446 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:40.621418 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:40.621588 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:40.621467 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:40.621660 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:40.621628 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:40.621931 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:40.621910 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:40.627254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:40.627235 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:41.408823 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:41.408787 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:41.413357 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:41.413335 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:29:41.662135 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:29:41.662040 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:30:43.105229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.105187 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr"] Aug 05 12:30:43.105761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.105467 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="online" containerID="cri-o://e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd" gracePeriod=30 Aug 05 12:30:43.105761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.105547 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="ui" containerID="cri-o://3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9" gracePeriod=30 Aug 05 12:30:43.106389 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.105627 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="offline" containerID="cri-o://70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4" gracePeriod=30 Aug 05 12:30:43.109243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.109169 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:30:43.225214 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.225175 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27"] Aug 05 12:30:43.225644 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.225458 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="registry" containerID="cri-o://a3c79cab4ef9d7c7fb736fd4d0dfdbda387fe015299f0c5ec55de7181c23db0b" gracePeriod=30 Aug 05 12:30:43.225644 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.225507 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="ui" containerID="cri-o://8435f201d1312532c6665c3cf219a5c3fb329e3504848d23445b0c54b52dff53" gracePeriod=30 Aug 05 12:30:43.225644 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.225510 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="online" containerID="cri-o://9a366d0dc5aa4be573d7e1a959690ac9348268d152a1ae23c9138fb7579599a7" gracePeriod=30 Aug 05 12:30:43.225644 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.225548 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="offline" containerID="cri-o://fb87c79886c30d78d8fee5be53ab2a0fbebdeddd62fac90868c0da0e885cf97a" gracePeriod=30 Aug 05 12:30:43.684081 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.684042 2569 generic.go:356] "Generic (PLEG): container finished" podID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerID="3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9" exitCode=0 Aug 05 12:30:43.684235 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:43.684131 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerDied","Data":"3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9"} Aug 05 12:30:44.691411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:44.691372 2569 generic.go:356] "Generic (PLEG): container finished" podID="76e37654-2f33-4479-911c-2616f6aadc09" containerID="8435f201d1312532c6665c3cf219a5c3fb329e3504848d23445b0c54b52dff53" exitCode=0 Aug 05 12:30:44.691411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:44.691400 2569 generic.go:356] "Generic (PLEG): container finished" podID="76e37654-2f33-4479-911c-2616f6aadc09" containerID="9a366d0dc5aa4be573d7e1a959690ac9348268d152a1ae23c9138fb7579599a7" exitCode=0 Aug 05 12:30:44.691823 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:44.691445 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerDied","Data":"8435f201d1312532c6665c3cf219a5c3fb329e3504848d23445b0c54b52dff53"} Aug 05 12:30:44.691823 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:44.691486 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerDied","Data":"9a366d0dc5aa4be573d7e1a959690ac9348268d152a1ae23c9138fb7579599a7"} Aug 05 12:30:44.693987 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:44.693964 2569 generic.go:356] "Generic (PLEG): container finished" podID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerID="e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd" exitCode=0 Aug 05 12:30:44.694099 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:44.694039 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerDied","Data":"e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd"} Aug 05 12:30:44.715761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:44.715734 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:30:48.392233 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:48.392142 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Aug 05 12:30:48.401467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:48.401442 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:30:49.038120 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:49.038081 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Aug 05 12:30:50.622116 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:50.622078 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Aug 05 12:30:51.409172 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:51.409131 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Aug 05 12:30:58.392236 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:58.392195 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Aug 05 12:30:59.037699 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:30:59.037658 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Aug 05 12:31:00.621679 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:00.621637 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Aug 05 12:31:01.409442 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:01.409394 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Aug 05 12:31:08.392420 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:08.392371 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Aug 05 12:31:08.392986 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:08.392520 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:31:09.038226 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:09.038182 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Aug 05 12:31:09.038419 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:09.038333 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:31:10.621504 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:10.621456 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.32:8443: connect: connection refused" Aug 05 12:31:10.621919 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:10.621589 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:31:11.409396 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:11.409357 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="online" probeResult="failure" output="Get \"https://10.132.0.32:6567/health\": dial tcp 10.132.0.32:6567: connect: connection refused" Aug 05 12:31:11.409570 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:11.409472 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:31:13.749132 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.749107 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:31:13.837064 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.837035 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-online-tls\") pod \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " Aug 05 12:31:13.837227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.837072 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-offline-tls\") pod \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " Aug 05 12:31:13.837227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.837105 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv6l6\" (UniqueName: \"kubernetes.io/projected/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-kube-api-access-jv6l6\") pod \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " Aug 05 12:31:13.837227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.837187 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-feast-data\") pod \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " Aug 05 12:31:13.837227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.837217 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-registry-tls\") pod \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " Aug 05 12:31:13.837451 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.837260 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls\") pod \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\" (UID: \"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9\") " Aug 05 12:31:13.838311 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.838052 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-feast-data" (OuterVolumeSpecName: "feast-data") pod "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" (UID: "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:31:13.838311 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.838195 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" (UID: "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:31:13.838655 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.838625 2569 generic.go:356] "Generic (PLEG): container finished" podID="76e37654-2f33-4479-911c-2616f6aadc09" containerID="fb87c79886c30d78d8fee5be53ab2a0fbebdeddd62fac90868c0da0e885cf97a" exitCode=137 Aug 05 12:31:13.838738 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.838655 2569 generic.go:356] "Generic (PLEG): container finished" podID="76e37654-2f33-4479-911c-2616f6aadc09" containerID="a3c79cab4ef9d7c7fb736fd4d0dfdbda387fe015299f0c5ec55de7181c23db0b" exitCode=137 Aug 05 12:31:13.838738 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.838708 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerDied","Data":"fb87c79886c30d78d8fee5be53ab2a0fbebdeddd62fac90868c0da0e885cf97a"} Aug 05 12:31:13.839313 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.839290 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerDied","Data":"a3c79cab4ef9d7c7fb736fd4d0dfdbda387fe015299f0c5ec55de7181c23db0b"} Aug 05 12:31:13.841534 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841505 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-online-tls" (OuterVolumeSpecName: "online-tls") pod "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" (UID: "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:31:13.841686 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841667 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" (UID: "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:31:13.841789 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841707 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-kube-api-access-jv6l6" (OuterVolumeSpecName: "kube-api-access-jv6l6") pod "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" (UID: "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9"). InnerVolumeSpecName "kube-api-access-jv6l6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:31:13.841789 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841697 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" (UID: "f69aa275-59ee-4a37-8a8e-1f0daf56cbd9"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:31:13.841789 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841759 2569 generic.go:356] "Generic (PLEG): container finished" podID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerID="70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4" exitCode=137 Aug 05 12:31:13.841915 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841851 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerDied","Data":"70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4"} Aug 05 12:31:13.841915 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841881 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" event={"ID":"f69aa275-59ee-4a37-8a8e-1f0daf56cbd9","Type":"ContainerDied","Data":"e719f49ce58129a0152380982cf949eeb185e95ec4cabe516794cfc5d9a48644"} Aug 05 12:31:13.841915 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841882 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr" Aug 05 12:31:13.841915 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.841898 2569 scope.go:117] "RemoveContainer" containerID="3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9" Aug 05 12:31:13.850732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.850716 2569 scope.go:117] "RemoveContainer" containerID="70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4" Aug 05 12:31:13.858535 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.858511 2569 scope.go:117] "RemoveContainer" containerID="e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd" Aug 05 12:31:13.868926 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.868909 2569 scope.go:117] "RemoveContainer" containerID="d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9" Aug 05 12:31:13.870792 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.870772 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr"] Aug 05 12:31:13.872429 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.872384 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6dccd445d5-6mtxr"] Aug 05 12:31:13.878216 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.878200 2569 scope.go:117] "RemoveContainer" containerID="0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5" Aug 05 12:31:13.881819 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.881800 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:31:13.885246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.885229 2569 scope.go:117] "RemoveContainer" containerID="3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9" Aug 05 12:31:13.885486 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:31:13.885468 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9\": container with ID starting with 3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9 not found: ID does not exist" containerID="3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9" Aug 05 12:31:13.885559 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.885493 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9"} err="failed to get container status \"3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9\": rpc error: code = NotFound desc = could not find container \"3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9\": container with ID starting with 3f1775d76b982635d3e8fe8314d6896090fac10f38d9437e59c90316113217c9 not found: ID does not exist" Aug 05 12:31:13.885559 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.885510 2569 scope.go:117] "RemoveContainer" containerID="70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4" Aug 05 12:31:13.885768 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:31:13.885750 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4\": container with ID starting with 70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4 not found: ID does not exist" containerID="70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4" Aug 05 12:31:13.885831 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.885773 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4"} err="failed to get container status \"70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4\": rpc error: code = NotFound desc = could not find container \"70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4\": container with ID starting with 70aafaf1af960d230c52f7ce40e0cb747135dbda8419d79970a2e99cf06033b4 not found: ID does not exist" Aug 05 12:31:13.885831 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.885791 2569 scope.go:117] "RemoveContainer" containerID="e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd" Aug 05 12:31:13.886043 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:31:13.886029 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd\": container with ID starting with e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd not found: ID does not exist" containerID="e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd" Aug 05 12:31:13.886100 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.886045 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd"} err="failed to get container status \"e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd\": rpc error: code = NotFound desc = could not find container \"e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd\": container with ID starting with e91c17cb9b3982b17ccc738803972507fd0743a49dd8f3d14b213271176270bd not found: ID does not exist" Aug 05 12:31:13.886100 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.886057 2569 scope.go:117] "RemoveContainer" containerID="d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9" Aug 05 12:31:13.886272 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:31:13.886257 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9\": container with ID starting with d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9 not found: ID does not exist" containerID="d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9" Aug 05 12:31:13.886311 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.886276 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9"} err="failed to get container status \"d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9\": rpc error: code = NotFound desc = could not find container \"d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9\": container with ID starting with d9cbe7db2152f54874795414928812b32c8f9d8e5c7e35809402225fb1264ac9 not found: ID does not exist" Aug 05 12:31:13.886311 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.886290 2569 scope.go:117] "RemoveContainer" containerID="0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5" Aug 05 12:31:13.886488 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:31:13.886470 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5\": container with ID starting with 0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5 not found: ID does not exist" containerID="0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5" Aug 05 12:31:13.886540 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.886496 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5"} err="failed to get container status \"0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5\": rpc error: code = NotFound desc = could not find container \"0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5\": container with ID starting with 0a3fc0e08139587b2c003fa8e2174f20967a784c8cd79478b695631476eddcc5 not found: ID does not exist" Aug 05 12:31:13.937825 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.937807 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/76e37654-2f33-4479-911c-2616f6aadc09-feast-data\") pod \"76e37654-2f33-4479-911c-2616f6aadc09\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " Aug 05 12:31:13.937928 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.937834 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-offline-tls\") pod \"76e37654-2f33-4479-911c-2616f6aadc09\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " Aug 05 12:31:13.937928 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.937853 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-registry-tls\") pod \"76e37654-2f33-4479-911c-2616f6aadc09\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " Aug 05 12:31:13.937928 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.937890 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-ui-tls\") pod \"76e37654-2f33-4479-911c-2616f6aadc09\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " Aug 05 12:31:13.938111 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.937927 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-online-tls\") pod \"76e37654-2f33-4479-911c-2616f6aadc09\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " Aug 05 12:31:13.938111 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.938036 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb8s8\" (UniqueName: \"kubernetes.io/projected/76e37654-2f33-4479-911c-2616f6aadc09-kube-api-access-jb8s8\") pod \"76e37654-2f33-4479-911c-2616f6aadc09\" (UID: \"76e37654-2f33-4479-911c-2616f6aadc09\") " Aug 05 12:31:13.938289 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.938272 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv6l6\" (UniqueName: \"kubernetes.io/projected/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-kube-api-access-jv6l6\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:13.938353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.938296 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:13.938353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.938310 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:13.938353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.938321 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:13.938353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.938333 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:13.938353 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.938345 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:13.938565 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.938373 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76e37654-2f33-4479-911c-2616f6aadc09-feast-data" (OuterVolumeSpecName: "feast-data") pod "76e37654-2f33-4479-911c-2616f6aadc09" (UID: "76e37654-2f33-4479-911c-2616f6aadc09"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:31:13.939915 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.939896 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-online-tls" (OuterVolumeSpecName: "online-tls") pod "76e37654-2f33-4479-911c-2616f6aadc09" (UID: "76e37654-2f33-4479-911c-2616f6aadc09"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:31:13.940344 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.940316 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "76e37654-2f33-4479-911c-2616f6aadc09" (UID: "76e37654-2f33-4479-911c-2616f6aadc09"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:31:13.940405 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.940346 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76e37654-2f33-4479-911c-2616f6aadc09-kube-api-access-jb8s8" (OuterVolumeSpecName: "kube-api-access-jb8s8") pod "76e37654-2f33-4479-911c-2616f6aadc09" (UID: "76e37654-2f33-4479-911c-2616f6aadc09"). InnerVolumeSpecName "kube-api-access-jb8s8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:31:13.940405 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.940376 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "76e37654-2f33-4479-911c-2616f6aadc09" (UID: "76e37654-2f33-4479-911c-2616f6aadc09"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:31:13.940471 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:13.940395 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "76e37654-2f33-4479-911c-2616f6aadc09" (UID: "76e37654-2f33-4479-911c-2616f6aadc09"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:31:14.039556 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.039532 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb8s8\" (UniqueName: \"kubernetes.io/projected/76e37654-2f33-4479-911c-2616f6aadc09-kube-api-access-jb8s8\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:14.039556 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.039555 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/76e37654-2f33-4479-911c-2616f6aadc09-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:14.039683 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.039565 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:14.039683 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.039573 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:14.039683 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.039581 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:14.039683 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.039591 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/76e37654-2f33-4479-911c-2616f6aadc09-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:31:14.849348 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.849308 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" event={"ID":"76e37654-2f33-4479-911c-2616f6aadc09","Type":"ContainerDied","Data":"8075afe3357ce83a75579a67f400620c6b12498dab1002fc04a5aa3479c2d970"} Aug 05 12:31:14.849779 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.849321 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27" Aug 05 12:31:14.849779 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.849365 2569 scope.go:117] "RemoveContainer" containerID="8435f201d1312532c6665c3cf219a5c3fb329e3504848d23445b0c54b52dff53" Aug 05 12:31:14.858629 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.858610 2569 scope.go:117] "RemoveContainer" containerID="fb87c79886c30d78d8fee5be53ab2a0fbebdeddd62fac90868c0da0e885cf97a" Aug 05 12:31:14.866182 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.866167 2569 scope.go:117] "RemoveContainer" containerID="9a366d0dc5aa4be573d7e1a959690ac9348268d152a1ae23c9138fb7579599a7" Aug 05 12:31:14.872109 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.872086 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27"] Aug 05 12:31:14.874723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.874697 2569 scope.go:117] "RemoveContainer" containerID="a3c79cab4ef9d7c7fb736fd4d0dfdbda387fe015299f0c5ec55de7181c23db0b" Aug 05 12:31:14.875552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.875534 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-6b89b8d8fb-nxr27"] Aug 05 12:31:14.881781 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.881766 2569 scope.go:117] "RemoveContainer" containerID="db405fbf93e37e3f1906ef4e3802b1bc0ce708b180ee6e5ae581eadbe9ced469" Aug 05 12:31:14.888771 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:14.888752 2569 scope.go:117] "RemoveContainer" containerID="1ab490708e791cf3137f6381a4cc7c0e99b21d932a16f75458a4e6b81d5a520f" Aug 05 12:31:15.441869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:15.441837 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76e37654-2f33-4479-911c-2616f6aadc09" path="/var/lib/kubelet/pods/76e37654-2f33-4479-911c-2616f6aadc09/volumes" Aug 05 12:31:15.442479 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:15.442465 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" path="/var/lib/kubelet/pods/f69aa275-59ee-4a37-8a8e-1f0daf56cbd9/volumes" Aug 05 12:31:25.970114 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:25.970077 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:31:26.172025 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.171993 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-t64nh"] Aug 05 12:31:26.172453 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172425 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="feast-apply" Aug 05 12:31:26.172453 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172451 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="feast-apply" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172465 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="registry" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172474 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="registry" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172495 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="online" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172503 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="online" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172514 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="feast-apply" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172523 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="feast-apply" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172537 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="ui" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172545 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="ui" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172558 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="ui" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172566 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="ui" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172578 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="feast-init" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172587 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="feast-init" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172598 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="feast-init" Aug 05 12:31:26.172605 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172606 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="feast-init" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172619 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="offline" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172627 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="offline" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172636 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="online" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172645 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="online" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172662 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="offline" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172670 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="offline" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172742 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="offline" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172754 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="online" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172763 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="registry" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172771 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="online" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172779 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="offline" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172787 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f69aa275-59ee-4a37-8a8e-1f0daf56cbd9" containerName="ui" Aug 05 12:31:26.173039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.172796 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e37654-2f33-4479-911c-2616f6aadc09" containerName="ui" Aug 05 12:31:26.176036 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.176018 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:31:26.178229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.178206 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Aug 05 12:31:26.178229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.178224 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Aug 05 12:31:26.178402 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.178267 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-b845v\"" Aug 05 12:31:26.178807 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.178793 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Aug 05 12:31:26.183642 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.183497 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-t64nh"] Aug 05 12:31:26.203254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.203231 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-24qkm"] Aug 05 12:31:26.206552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.206530 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-24qkm" Aug 05 12:31:26.209012 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.208988 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:31:26.213901 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.213876 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-24qkm"] Aug 05 12:31:26.234363 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.234312 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a50562d7-65dd-4f82-af25-a6a961cd5c70-postgresdata\") pod \"postgres-864846d4f5-t64nh\" (UID: \"a50562d7-65dd-4f82-af25-a6a961cd5c70\") " pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:31:26.234363 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.234359 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66sxh\" (UniqueName: \"kubernetes.io/projected/a50562d7-65dd-4f82-af25-a6a961cd5c70-kube-api-access-66sxh\") pod \"postgres-864846d4f5-t64nh\" (UID: \"a50562d7-65dd-4f82-af25-a6a961cd5c70\") " pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:31:26.234522 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.234442 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg469\" (UniqueName: \"kubernetes.io/projected/88d507da-7284-4587-96dc-4d3ab92709c8-kube-api-access-zg469\") pod \"redis-69bbd798b6-24qkm\" (UID: \"88d507da-7284-4587-96dc-4d3ab92709c8\") " pod="test-ns-feast/redis-69bbd798b6-24qkm" Aug 05 12:31:26.264339 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.264314 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:31:26.335272 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.335252 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66sxh\" (UniqueName: \"kubernetes.io/projected/a50562d7-65dd-4f82-af25-a6a961cd5c70-kube-api-access-66sxh\") pod \"postgres-864846d4f5-t64nh\" (UID: \"a50562d7-65dd-4f82-af25-a6a961cd5c70\") " pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:31:26.335370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.335294 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg469\" (UniqueName: \"kubernetes.io/projected/88d507da-7284-4587-96dc-4d3ab92709c8-kube-api-access-zg469\") pod \"redis-69bbd798b6-24qkm\" (UID: \"88d507da-7284-4587-96dc-4d3ab92709c8\") " pod="test-ns-feast/redis-69bbd798b6-24qkm" Aug 05 12:31:26.335370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.335343 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a50562d7-65dd-4f82-af25-a6a961cd5c70-postgresdata\") pod \"postgres-864846d4f5-t64nh\" (UID: \"a50562d7-65dd-4f82-af25-a6a961cd5c70\") " pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:31:26.335656 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.335642 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a50562d7-65dd-4f82-af25-a6a961cd5c70-postgresdata\") pod \"postgres-864846d4f5-t64nh\" (UID: \"a50562d7-65dd-4f82-af25-a6a961cd5c70\") " pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:31:26.343471 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.343453 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg469\" (UniqueName: \"kubernetes.io/projected/88d507da-7284-4587-96dc-4d3ab92709c8-kube-api-access-zg469\") pod \"redis-69bbd798b6-24qkm\" (UID: \"88d507da-7284-4587-96dc-4d3ab92709c8\") " pod="test-ns-feast/redis-69bbd798b6-24qkm" Aug 05 12:31:26.343555 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.343454 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66sxh\" (UniqueName: \"kubernetes.io/projected/a50562d7-65dd-4f82-af25-a6a961cd5c70-kube-api-access-66sxh\") pod \"postgres-864846d4f5-t64nh\" (UID: \"a50562d7-65dd-4f82-af25-a6a961cd5c70\") " pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:31:26.493822 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.493772 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:31:26.518157 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.518134 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-24qkm" Aug 05 12:31:26.634254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.634224 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-t64nh"] Aug 05 12:31:26.635264 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:31:26.635239 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda50562d7_65dd_4f82_af25_a6a961cd5c70.slice/crio-168770a3d35f9f970346a265834692703c960c18f060979ab116d90553260c88 WatchSource:0}: Error finding container 168770a3d35f9f970346a265834692703c960c18f060979ab116d90553260c88: Status 404 returned error can't find the container with id 168770a3d35f9f970346a265834692703c960c18f060979ab116d90553260c88 Aug 05 12:31:26.637628 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.637609 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 12:31:26.656232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.656209 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-24qkm"] Aug 05 12:31:26.658077 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:31:26.658054 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88d507da_7284_4587_96dc_4d3ab92709c8.slice/crio-452e7fece9aef2a7cf1b2b15750533379dbe48a1741f33b8683ca32c76ad1021 WatchSource:0}: Error finding container 452e7fece9aef2a7cf1b2b15750533379dbe48a1741f33b8683ca32c76ad1021: Status 404 returned error can't find the container with id 452e7fece9aef2a7cf1b2b15750533379dbe48a1741f33b8683ca32c76ad1021 Aug 05 12:31:26.903870 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.903841 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-24qkm" event={"ID":"88d507da-7284-4587-96dc-4d3ab92709c8","Type":"ContainerStarted","Data":"452e7fece9aef2a7cf1b2b15750533379dbe48a1741f33b8683ca32c76ad1021"} Aug 05 12:31:26.904970 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:26.904931 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-t64nh" event={"ID":"a50562d7-65dd-4f82-af25-a6a961cd5c70","Type":"ContainerStarted","Data":"168770a3d35f9f970346a265834692703c960c18f060979ab116d90553260c88"} Aug 05 12:31:28.917529 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:28.917494 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-24qkm" event={"ID":"88d507da-7284-4587-96dc-4d3ab92709c8","Type":"ContainerStarted","Data":"5d2e2c2af5523419328850d06021c840b02203d66a7da6b0a7e536f7bfa37638"} Aug 05 12:31:28.933399 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:28.933349 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-24qkm" podStartSLOduration=1.213020054 podStartE2EDuration="2.933335523s" totalImagesPullingTime="1.720315469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:31:28.932150661 +0000 UTC m=+753.997742560" watchObservedRunningTime="2026-08-05 12:31:28.933335523 +0000 UTC m=+753.998927421" Aug 05 12:31:32.941886 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:32.941853 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-t64nh" event={"ID":"a50562d7-65dd-4f82-af25-a6a961cd5c70","Type":"ContainerStarted","Data":"142e247038b2f02264d705f7f105d4ba7d5c4c0c1e358d3ff5d9f30f8ed59858"} Aug 05 12:31:32.957399 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:32.957354 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-t64nh" podStartSLOduration=1.675234583 podStartE2EDuration="6.957339935s" totalImagesPullingTime="5.282105352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:31:32.956260429 +0000 UTC m=+758.021852361" watchObservedRunningTime="2026-08-05 12:31:32.957339935 +0000 UTC m=+758.022931832" Aug 05 12:31:36.710934 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.710882 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:31:36.810942 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.810914 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:31:36.820425 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.820401 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-87c965d76-2h58h"] Aug 05 12:31:36.827777 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.827759 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:36.830199 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.830175 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Aug 05 12:31:36.830199 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.830190 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Aug 05 12:31:36.830354 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.830219 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Aug 05 12:31:36.830479 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.830467 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-ww2d2\"" Aug 05 12:31:36.830479 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.830473 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Aug 05 12:31:36.835854 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.835832 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-87c965d76-2h58h"] Aug 05 12:31:36.926712 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.926688 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:36.926837 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.926717 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-offline-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:36.926837 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.926738 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-online-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:36.926837 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.926796 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf997\" (UniqueName: \"kubernetes.io/projected/7a5961e2-2427-4ef6-99f1-883b7a3f0172-kube-api-access-tf997\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:36.926837 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.926837 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7a5961e2-2427-4ef6-99f1-883b7a3f0172-feast-data\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:36.997303 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:36.997237 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:31:37.027196 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.027174 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7a5961e2-2427-4ef6-99f1-883b7a3f0172-feast-data\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.027292 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.027204 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.027292 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.027223 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-offline-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.027393 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:31:37.027299 2569 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Aug 05 12:31:37.027393 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.027336 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-online-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.027393 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:31:37.027352 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls podName:7a5961e2-2427-4ef6-99f1-883b7a3f0172 nodeName:}" failed. No retries permitted until 2026-08-05 12:31:37.527337685 +0000 UTC m=+762.592929562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls") pod "feast-credit-scoring-87c965d76-2h58h" (UID: "7a5961e2-2427-4ef6-99f1-883b7a3f0172") : secret "feast-credit-scoring-registry-tls" not found Aug 05 12:31:37.027547 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.027400 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf997\" (UniqueName: \"kubernetes.io/projected/7a5961e2-2427-4ef6-99f1-883b7a3f0172-kube-api-access-tf997\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.027602 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.027585 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7a5961e2-2427-4ef6-99f1-883b7a3f0172-feast-data\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.029671 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.029650 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-offline-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.029758 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.029725 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-online-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.035273 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.035245 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf997\" (UniqueName: \"kubernetes.io/projected/7a5961e2-2427-4ef6-99f1-883b7a3f0172-kube-api-access-tf997\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.097962 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.097923 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:31:37.531885 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.531854 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.534263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.534239 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls\") pod \"feast-credit-scoring-87c965d76-2h58h\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.592416 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.592388 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:31:37.737514 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.737488 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:37.870700 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.870675 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-87c965d76-2h58h"] Aug 05 12:31:37.872124 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:31:37.872098 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a5961e2_2427_4ef6_99f1_883b7a3f0172.slice/crio-357fda9ef438e68602c48f9e67c4e10cc8f76d2572ef8badb6c9322ad4c014ea WatchSource:0}: Error finding container 357fda9ef438e68602c48f9e67c4e10cc8f76d2572ef8badb6c9322ad4c014ea: Status 404 returned error can't find the container with id 357fda9ef438e68602c48f9e67c4e10cc8f76d2572ef8badb6c9322ad4c014ea Aug 05 12:31:37.964772 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.964745 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerStarted","Data":"003d766a7195a8883a3e7c4c8c95cfa32d0c9bba3bb23cdfc13cb0102e245635"} Aug 05 12:31:37.964933 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:37.964812 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerStarted","Data":"357fda9ef438e68602c48f9e67c4e10cc8f76d2572ef8badb6c9322ad4c014ea"} Aug 05 12:31:39.975044 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:39.975015 2569 generic.go:356] "Generic (PLEG): container finished" podID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerID="003d766a7195a8883a3e7c4c8c95cfa32d0c9bba3bb23cdfc13cb0102e245635" exitCode=0 Aug 05 12:31:39.975339 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:39.975090 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerDied","Data":"003d766a7195a8883a3e7c4c8c95cfa32d0c9bba3bb23cdfc13cb0102e245635"} Aug 05 12:31:40.982370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:40.982330 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerStarted","Data":"ca92792f533ba2ef73aaa76afa5c19def9e6030b24693c10eda1acd1caa9885a"} Aug 05 12:31:47.011630 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:47.011595 2569 generic.go:356] "Generic (PLEG): container finished" podID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerID="ca92792f533ba2ef73aaa76afa5c19def9e6030b24693c10eda1acd1caa9885a" exitCode=0 Aug 05 12:31:47.012033 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:47.011666 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerDied","Data":"ca92792f533ba2ef73aaa76afa5c19def9e6030b24693c10eda1acd1caa9885a"} Aug 05 12:31:48.020715 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:48.020675 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerStarted","Data":"983b6d1dd5c1c6478ef61e0558e5588b640f4b7a4a7512390b0ea3f98771b025"} Aug 05 12:31:48.021190 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:48.020750 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerStarted","Data":"a5690bde942f9ab3a228864aff5d4175471bb151381c52101dce99f266465a7d"} Aug 05 12:31:48.021190 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:48.020799 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerStarted","Data":"efb98c24a13305b17250996b00163a20007d2aa7acac351991ddd110a43c0ade"} Aug 05 12:31:48.047005 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:48.046930 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podStartSLOduration=12.046909557 podStartE2EDuration="12.046909557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:31:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:31:48.044939908 +0000 UTC m=+773.110531807" watchObservedRunningTime="2026-08-05 12:31:48.046909557 +0000 UTC m=+773.112501457" Aug 05 12:31:49.737914 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:49.737871 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:49.738409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:49.737975 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:49.738409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:49.738003 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:49.739375 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:49.739242 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.35:8816: connect: connection refused" Aug 05 12:31:49.739375 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:49.739276 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Aug 05 12:31:49.739375 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:49.739286 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Aug 05 12:31:52.739017 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:52.738943 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Aug 05 12:31:52.739619 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:52.738943 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Aug 05 12:31:52.739619 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:52.739506 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:52.739974 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:52.739899 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:52.740481 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:52.740458 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:55.738156 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:55.738060 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Aug 05 12:31:55.743263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:55.743241 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:55.743720 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:55.743695 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:55.746762 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:55.746744 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:57.738704 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:57.738668 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:58.739218 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:58.739187 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:59.080389 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:59.080366 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:31:59.322358 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:31:59.322317 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:32:57.231827 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:32:57.231789 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:33:02.218526 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.218494 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-87c965d76-2h58h"] Aug 05 12:33:02.218989 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.218724 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="registry" containerID="cri-o://efb98c24a13305b17250996b00163a20007d2aa7acac351991ddd110a43c0ade" gracePeriod=30 Aug 05 12:33:02.218989 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.218776 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" containerID="cri-o://a5690bde942f9ab3a228864aff5d4175471bb151381c52101dce99f266465a7d" gracePeriod=30 Aug 05 12:33:02.218989 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.218784 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="offline" containerID="cri-o://983b6d1dd5c1c6478ef61e0558e5588b640f4b7a4a7512390b0ea3f98771b025" gracePeriod=30 Aug 05 12:33:02.238373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.238345 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-t64nh"] Aug 05 12:33:02.238540 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.238520 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-t64nh" podUID="a50562d7-65dd-4f82-af25-a6a961cd5c70" containerName="postgres" containerID="cri-o://142e247038b2f02264d705f7f105d4ba7d5c4c0c1e358d3ff5d9f30f8ed59858" gracePeriod=30 Aug 05 12:33:02.244316 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.244288 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-24qkm"] Aug 05 12:33:02.244478 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.244460 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-24qkm" podUID="88d507da-7284-4587-96dc-4d3ab92709c8" containerName="redis" containerID="cri-o://5d2e2c2af5523419328850d06021c840b02203d66a7da6b0a7e536f7bfa37638" gracePeriod=30 Aug 05 12:33:02.300272 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.300239 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:33:02.375559 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.375529 2569 generic.go:356] "Generic (PLEG): container finished" podID="88d507da-7284-4587-96dc-4d3ab92709c8" containerID="5d2e2c2af5523419328850d06021c840b02203d66a7da6b0a7e536f7bfa37638" exitCode=0 Aug 05 12:33:02.375727 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.375673 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-24qkm" event={"ID":"88d507da-7284-4587-96dc-4d3ab92709c8","Type":"ContainerDied","Data":"5d2e2c2af5523419328850d06021c840b02203d66a7da6b0a7e536f7bfa37638"} Aug 05 12:33:02.377769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.377736 2569 generic.go:356] "Generic (PLEG): container finished" podID="a50562d7-65dd-4f82-af25-a6a961cd5c70" containerID="142e247038b2f02264d705f7f105d4ba7d5c4c0c1e358d3ff5d9f30f8ed59858" exitCode=0 Aug 05 12:33:02.377997 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.377976 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-t64nh" event={"ID":"a50562d7-65dd-4f82-af25-a6a961cd5c70","Type":"ContainerDied","Data":"142e247038b2f02264d705f7f105d4ba7d5c4c0c1e358d3ff5d9f30f8ed59858"} Aug 05 12:33:02.504812 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.504777 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:33:02.507941 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.507919 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-24qkm" Aug 05 12:33:02.553859 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.553823 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg469\" (UniqueName: \"kubernetes.io/projected/88d507da-7284-4587-96dc-4d3ab92709c8-kube-api-access-zg469\") pod \"88d507da-7284-4587-96dc-4d3ab92709c8\" (UID: \"88d507da-7284-4587-96dc-4d3ab92709c8\") " Aug 05 12:33:02.554058 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.553897 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66sxh\" (UniqueName: \"kubernetes.io/projected/a50562d7-65dd-4f82-af25-a6a961cd5c70-kube-api-access-66sxh\") pod \"a50562d7-65dd-4f82-af25-a6a961cd5c70\" (UID: \"a50562d7-65dd-4f82-af25-a6a961cd5c70\") " Aug 05 12:33:02.554058 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.554045 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a50562d7-65dd-4f82-af25-a6a961cd5c70-postgresdata\") pod \"a50562d7-65dd-4f82-af25-a6a961cd5c70\" (UID: \"a50562d7-65dd-4f82-af25-a6a961cd5c70\") " Aug 05 12:33:02.556473 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.556442 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88d507da-7284-4587-96dc-4d3ab92709c8-kube-api-access-zg469" (OuterVolumeSpecName: "kube-api-access-zg469") pod "88d507da-7284-4587-96dc-4d3ab92709c8" (UID: "88d507da-7284-4587-96dc-4d3ab92709c8"). InnerVolumeSpecName "kube-api-access-zg469". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:33:02.556589 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.556501 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50562d7-65dd-4f82-af25-a6a961cd5c70-kube-api-access-66sxh" (OuterVolumeSpecName: "kube-api-access-66sxh") pod "a50562d7-65dd-4f82-af25-a6a961cd5c70" (UID: "a50562d7-65dd-4f82-af25-a6a961cd5c70"). InnerVolumeSpecName "kube-api-access-66sxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:33:02.586747 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.586713 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50562d7-65dd-4f82-af25-a6a961cd5c70-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "a50562d7-65dd-4f82-af25-a6a961cd5c70" (UID: "a50562d7-65dd-4f82-af25-a6a961cd5c70"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:33:02.654708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.654665 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66sxh\" (UniqueName: \"kubernetes.io/projected/a50562d7-65dd-4f82-af25-a6a961cd5c70-kube-api-access-66sxh\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:33:02.654708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.654694 2569 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/a50562d7-65dd-4f82-af25-a6a961cd5c70-postgresdata\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:33:02.654708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:02.654704 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg469\" (UniqueName: \"kubernetes.io/projected/88d507da-7284-4587-96dc-4d3ab92709c8-kube-api-access-zg469\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:33:03.383237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.383203 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-24qkm" Aug 05 12:33:03.383237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.383211 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-24qkm" event={"ID":"88d507da-7284-4587-96dc-4d3ab92709c8","Type":"ContainerDied","Data":"452e7fece9aef2a7cf1b2b15750533379dbe48a1741f33b8683ca32c76ad1021"} Aug 05 12:33:03.383762 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.383261 2569 scope.go:117] "RemoveContainer" containerID="5d2e2c2af5523419328850d06021c840b02203d66a7da6b0a7e536f7bfa37638" Aug 05 12:33:03.386087 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.386062 2569 generic.go:356] "Generic (PLEG): container finished" podID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerID="a5690bde942f9ab3a228864aff5d4175471bb151381c52101dce99f266465a7d" exitCode=0 Aug 05 12:33:03.386211 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.386132 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerDied","Data":"a5690bde942f9ab3a228864aff5d4175471bb151381c52101dce99f266465a7d"} Aug 05 12:33:03.387428 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.387404 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-t64nh" event={"ID":"a50562d7-65dd-4f82-af25-a6a961cd5c70","Type":"ContainerDied","Data":"168770a3d35f9f970346a265834692703c960c18f060979ab116d90553260c88"} Aug 05 12:33:03.387428 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.387427 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-t64nh" Aug 05 12:33:03.392830 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.392811 2569 scope.go:117] "RemoveContainer" containerID="142e247038b2f02264d705f7f105d4ba7d5c4c0c1e358d3ff5d9f30f8ed59858" Aug 05 12:33:03.405417 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.405397 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-24qkm"] Aug 05 12:33:03.408817 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.408797 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-24qkm"] Aug 05 12:33:03.418874 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.418836 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-t64nh"] Aug 05 12:33:03.420424 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.420404 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-t64nh"] Aug 05 12:33:03.441159 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.441140 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88d507da-7284-4587-96dc-4d3ab92709c8" path="/var/lib/kubelet/pods/88d507da-7284-4587-96dc-4d3ab92709c8/volumes" Aug 05 12:33:03.441467 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:03.441455 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a50562d7-65dd-4f82-af25-a6a961cd5c70" path="/var/lib/kubelet/pods/a50562d7-65dd-4f82-af25-a6a961cd5c70/volumes" Aug 05 12:33:05.744318 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:05.744264 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Aug 05 12:33:15.744033 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:15.743989 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Aug 05 12:33:25.744550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:25.744509 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Aug 05 12:33:25.745005 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:25.744618 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:33:32.522769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.522736 2569 generic.go:356] "Generic (PLEG): container finished" podID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerID="983b6d1dd5c1c6478ef61e0558e5588b640f4b7a4a7512390b0ea3f98771b025" exitCode=137 Aug 05 12:33:32.522769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.522760 2569 generic.go:356] "Generic (PLEG): container finished" podID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerID="efb98c24a13305b17250996b00163a20007d2aa7acac351991ddd110a43c0ade" exitCode=137 Aug 05 12:33:32.523216 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.522806 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerDied","Data":"983b6d1dd5c1c6478ef61e0558e5588b640f4b7a4a7512390b0ea3f98771b025"} Aug 05 12:33:32.523216 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.522883 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerDied","Data":"efb98c24a13305b17250996b00163a20007d2aa7acac351991ddd110a43c0ade"} Aug 05 12:33:32.741086 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.741054 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.35:8816: connect: connection refused" Aug 05 12:33:32.866925 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.866898 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:33:32.993234 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.993201 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-online-tls\") pod \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " Aug 05 12:33:32.993234 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.993238 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf997\" (UniqueName: \"kubernetes.io/projected/7a5961e2-2427-4ef6-99f1-883b7a3f0172-kube-api-access-tf997\") pod \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " Aug 05 12:33:32.993471 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.993269 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls\") pod \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " Aug 05 12:33:32.993471 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.993295 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7a5961e2-2427-4ef6-99f1-883b7a3f0172-feast-data\") pod \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " Aug 05 12:33:32.993471 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.993409 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-offline-tls\") pod \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\" (UID: \"7a5961e2-2427-4ef6-99f1-883b7a3f0172\") " Aug 05 12:33:32.995433 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.995402 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7a5961e2-2427-4ef6-99f1-883b7a3f0172" (UID: "7a5961e2-2427-4ef6-99f1-883b7a3f0172"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:33:32.995433 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.995419 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-online-tls" (OuterVolumeSpecName: "online-tls") pod "7a5961e2-2427-4ef6-99f1-883b7a3f0172" (UID: "7a5961e2-2427-4ef6-99f1-883b7a3f0172"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:33:32.995738 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.995713 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a5961e2-2427-4ef6-99f1-883b7a3f0172-kube-api-access-tf997" (OuterVolumeSpecName: "kube-api-access-tf997") pod "7a5961e2-2427-4ef6-99f1-883b7a3f0172" (UID: "7a5961e2-2427-4ef6-99f1-883b7a3f0172"). InnerVolumeSpecName "kube-api-access-tf997". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:33:32.995858 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:32.995739 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7a5961e2-2427-4ef6-99f1-883b7a3f0172" (UID: "7a5961e2-2427-4ef6-99f1-883b7a3f0172"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:33:33.001267 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.001247 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a5961e2-2427-4ef6-99f1-883b7a3f0172-feast-data" (OuterVolumeSpecName: "feast-data") pod "7a5961e2-2427-4ef6-99f1-883b7a3f0172" (UID: "7a5961e2-2427-4ef6-99f1-883b7a3f0172"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:33:33.094392 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.094310 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:33:33.094392 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.094339 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf997\" (UniqueName: \"kubernetes.io/projected/7a5961e2-2427-4ef6-99f1-883b7a3f0172-kube-api-access-tf997\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:33:33.094392 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.094351 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:33:33.094392 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.094360 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7a5961e2-2427-4ef6-99f1-883b7a3f0172-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:33:33.094392 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.094368 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7a5961e2-2427-4ef6-99f1-883b7a3f0172-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:33:33.530104 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.530056 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" event={"ID":"7a5961e2-2427-4ef6-99f1-883b7a3f0172","Type":"ContainerDied","Data":"357fda9ef438e68602c48f9e67c4e10cc8f76d2572ef8badb6c9322ad4c014ea"} Aug 05 12:33:33.530527 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.530125 2569 scope.go:117] "RemoveContainer" containerID="983b6d1dd5c1c6478ef61e0558e5588b640f4b7a4a7512390b0ea3f98771b025" Aug 05 12:33:33.530527 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.530074 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-87c965d76-2h58h" Aug 05 12:33:33.538666 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.538618 2569 scope.go:117] "RemoveContainer" containerID="a5690bde942f9ab3a228864aff5d4175471bb151381c52101dce99f266465a7d" Aug 05 12:33:33.546276 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.546251 2569 scope.go:117] "RemoveContainer" containerID="efb98c24a13305b17250996b00163a20007d2aa7acac351991ddd110a43c0ade" Aug 05 12:33:33.549065 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.549043 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-87c965d76-2h58h"] Aug 05 12:33:33.551915 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.551890 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-87c965d76-2h58h"] Aug 05 12:33:33.554461 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.554446 2569 scope.go:117] "RemoveContainer" containerID="ca92792f533ba2ef73aaa76afa5c19def9e6030b24693c10eda1acd1caa9885a" Aug 05 12:33:33.561443 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:33.561427 2569 scope.go:117] "RemoveContainer" containerID="003d766a7195a8883a3e7c4c8c95cfa32d0c9bba3bb23cdfc13cb0102e245635" Aug 05 12:33:35.441919 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:35.441875 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" path="/var/lib/kubelet/pods/7a5961e2-2427-4ef6-99f1-883b7a3f0172/volumes" Aug 05 12:33:39.989912 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:39.989871 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:33:40.051672 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:40.051639 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:33:40.425221 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:40.425187 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:33:40.481250 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:40.481217 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:33:40.659364 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:40.659332 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:33:55.350415 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:55.350335 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:33:55.352523 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:33:55.352496 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:35:53.496345 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:35:53.496308 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:35:54.786994 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:35:54.786944 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:16.365098 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.365063 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:16.467097 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.467071 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:16.539495 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539468 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-wzwn4"] Aug 05 12:36:16.539797 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539784 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50562d7-65dd-4f82-af25-a6a961cd5c70" containerName="postgres" Aug 05 12:36:16.539846 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539799 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50562d7-65dd-4f82-af25-a6a961cd5c70" containerName="postgres" Aug 05 12:36:16.539846 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539809 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" Aug 05 12:36:16.539846 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539814 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" Aug 05 12:36:16.539846 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539825 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="feast-init" Aug 05 12:36:16.539846 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539831 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="feast-init" Aug 05 12:36:16.539846 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539840 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="offline" Aug 05 12:36:16.539846 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539845 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="offline" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539852 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="registry" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539857 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="registry" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539868 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="feast-apply" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539873 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="feast-apply" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539883 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88d507da-7284-4587-96dc-4d3ab92709c8" containerName="redis" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539890 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="88d507da-7284-4587-96dc-4d3ab92709c8" containerName="redis" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539978 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="offline" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539988 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="registry" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.539995 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50562d7-65dd-4f82-af25-a6a961cd5c70" containerName="postgres" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.540003 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="88d507da-7284-4587-96dc-4d3ab92709c8" containerName="redis" Aug 05 12:36:16.540091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.540008 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a5961e2-2427-4ef6-99f1-883b7a3f0172" containerName="online" Aug 05 12:36:16.542847 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.542825 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" Aug 05 12:36:16.545114 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.545089 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-ht2ml\"" Aug 05 12:36:16.545232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.545088 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Aug 05 12:36:16.545232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.545102 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Aug 05 12:36:16.558617 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.558598 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-wzwn4"] Aug 05 12:36:16.570625 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.570601 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:16.636884 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.636807 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9rb9\" (UniqueName: \"kubernetes.io/projected/f2f98ee6-b4fd-40a5-915b-fe912ae38702-kube-api-access-w9rb9\") pod \"redis-69bbd798b6-wzwn4\" (UID: \"f2f98ee6-b4fd-40a5-915b-fe912ae38702\") " pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" Aug 05 12:36:16.738241 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.738219 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9rb9\" (UniqueName: \"kubernetes.io/projected/f2f98ee6-b4fd-40a5-915b-fe912ae38702-kube-api-access-w9rb9\") pod \"redis-69bbd798b6-wzwn4\" (UID: \"f2f98ee6-b4fd-40a5-915b-fe912ae38702\") " pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" Aug 05 12:36:16.747066 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.747043 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9rb9\" (UniqueName: \"kubernetes.io/projected/f2f98ee6-b4fd-40a5-915b-fe912ae38702-kube-api-access-w9rb9\") pod \"redis-69bbd798b6-wzwn4\" (UID: \"f2f98ee6-b4fd-40a5-915b-fe912ae38702\") " pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" Aug 05 12:36:16.864867 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.864831 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" Aug 05 12:36:16.987579 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:16.987552 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-wzwn4"] Aug 05 12:36:16.990172 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:36:16.990145 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2f98ee6_b4fd_40a5_915b_fe912ae38702.slice/crio-3cbae61f8934b10299da290a711f95bed8745c4e000c01a15fa337e707860306 WatchSource:0}: Error finding container 3cbae61f8934b10299da290a711f95bed8745c4e000c01a15fa337e707860306: Status 404 returned error can't find the container with id 3cbae61f8934b10299da290a711f95bed8745c4e000c01a15fa337e707860306 Aug 05 12:36:17.180706 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:17.180639 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" event={"ID":"f2f98ee6-b4fd-40a5-915b-fe912ae38702","Type":"ContainerStarted","Data":"c76f4dbf728efe641f17e88eee99193039846279d4e40871cb7018ea5e3abe4e"} Aug 05 12:36:17.180706 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:17.180678 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" event={"ID":"f2f98ee6-b4fd-40a5-915b-fe912ae38702","Type":"ContainerStarted","Data":"3cbae61f8934b10299da290a711f95bed8745c4e000c01a15fa337e707860306"} Aug 05 12:36:17.195550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:17.195502 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" podStartSLOduration=1.195483172 podStartE2EDuration="1.195483172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:36:17.194700919 +0000 UTC m=+1042.260292817" watchObservedRunningTime="2026-08-05 12:36:17.195483172 +0000 UTC m=+1042.261075071" Aug 05 12:36:21.982153 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:21.982120 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nb2pm"] Aug 05 12:36:21.985717 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:21.985696 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:36:21.987661 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:21.987635 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Aug 05 12:36:21.992929 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:21.992906 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nb2pm"] Aug 05 12:36:22.018334 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.018309 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:22.080666 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.080640 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40a8a370-5e18-45e7-ad64-a21f197df3b0-postgresdata\") pod \"postgres-864846d4f5-nb2pm\" (UID: \"40a8a370-5e18-45e7-ad64-a21f197df3b0\") " pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:36:22.080756 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.080700 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2jrf\" (UniqueName: \"kubernetes.io/projected/40a8a370-5e18-45e7-ad64-a21f197df3b0-kube-api-access-m2jrf\") pod \"postgres-864846d4f5-nb2pm\" (UID: \"40a8a370-5e18-45e7-ad64-a21f197df3b0\") " pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:36:22.181730 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.181703 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40a8a370-5e18-45e7-ad64-a21f197df3b0-postgresdata\") pod \"postgres-864846d4f5-nb2pm\" (UID: \"40a8a370-5e18-45e7-ad64-a21f197df3b0\") " pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:36:22.181835 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.181743 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2jrf\" (UniqueName: \"kubernetes.io/projected/40a8a370-5e18-45e7-ad64-a21f197df3b0-kube-api-access-m2jrf\") pod \"postgres-864846d4f5-nb2pm\" (UID: \"40a8a370-5e18-45e7-ad64-a21f197df3b0\") " pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:36:22.182116 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.182099 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40a8a370-5e18-45e7-ad64-a21f197df3b0-postgresdata\") pod \"postgres-864846d4f5-nb2pm\" (UID: \"40a8a370-5e18-45e7-ad64-a21f197df3b0\") " pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:36:22.189513 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.189493 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2jrf\" (UniqueName: \"kubernetes.io/projected/40a8a370-5e18-45e7-ad64-a21f197df3b0-kube-api-access-m2jrf\") pod \"postgres-864846d4f5-nb2pm\" (UID: \"40a8a370-5e18-45e7-ad64-a21f197df3b0\") " pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:36:22.296517 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.296493 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:36:22.427401 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:22.427365 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nb2pm"] Aug 05 12:36:22.431033 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:36:22.431007 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40a8a370_5e18_45e7_ad64_a21f197df3b0.slice/crio-8839c2d21fd0f6c5d8037d4be4956933e9eb392332b8d932011ab633c19a733f WatchSource:0}: Error finding container 8839c2d21fd0f6c5d8037d4be4956933e9eb392332b8d932011ab633c19a733f: Status 404 returned error can't find the container with id 8839c2d21fd0f6c5d8037d4be4956933e9eb392332b8d932011ab633c19a733f Aug 05 12:36:23.206463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:23.206428 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" event={"ID":"40a8a370-5e18-45e7-ad64-a21f197df3b0","Type":"ContainerStarted","Data":"287ea3086d18411c0983d52d9eacf82d9f12d290ffed0b74565d90ef91d8fa1e"} Aug 05 12:36:23.206839 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:23.206470 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" event={"ID":"40a8a370-5e18-45e7-ad64-a21f197df3b0","Type":"ContainerStarted","Data":"8839c2d21fd0f6c5d8037d4be4956933e9eb392332b8d932011ab633c19a733f"} Aug 05 12:36:23.221937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:23.221890 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" podStartSLOduration=2.221876884 podStartE2EDuration="2.221876884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:36:23.220719268 +0000 UTC m=+1048.286311167" watchObservedRunningTime="2026-08-05 12:36:23.221876884 +0000 UTC m=+1048.287468783" Aug 05 12:36:27.519394 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.519358 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:27.595850 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.595796 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:27.596521 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.596502 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz"] Aug 05 12:36:27.599151 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.599132 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.601198 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.601174 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Aug 05 12:36:27.601313 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.601174 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Aug 05 12:36:27.601386 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.601318 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Aug 05 12:36:27.601482 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.601467 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-8f565\"" Aug 05 12:36:27.610821 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.610799 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz"] Aug 05 12:36:27.656691 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.656665 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:27.725792 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.725760 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.725905 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.725797 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrzs6\" (UniqueName: \"kubernetes.io/projected/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-kube-api-access-vrzs6\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.725905 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.725862 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-registry-tls\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.726042 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.725930 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-feast-data\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.827006 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.826912 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.827006 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.826945 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrzs6\" (UniqueName: \"kubernetes.io/projected/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-kube-api-access-vrzs6\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.827182 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.827021 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-registry-tls\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.827182 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:36:27.827054 2569 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Aug 05 12:36:27.827182 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.827088 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-feast-data\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.827182 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:36:27.827108 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls podName:e85b5332-1aa7-46b0-8f3c-159a6cf1d534 nodeName:}" failed. No retries permitted until 2026-08-05 12:36:28.327092813 +0000 UTC m=+1053.392684689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls") pod "feast-credit-scoring-985d988fc-r8shz" (UID: "e85b5332-1aa7-46b0-8f3c-159a6cf1d534") : secret "feast-credit-scoring-online-tls" not found Aug 05 12:36:27.827472 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.827453 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-feast-data\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.829338 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.829320 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-registry-tls\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.837486 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.837461 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrzs6\" (UniqueName: \"kubernetes.io/projected/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-kube-api-access-vrzs6\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:27.902063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:27.902036 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:28.330816 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:28.330782 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:28.333215 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:28.333183 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls\") pod \"feast-credit-scoring-985d988fc-r8shz\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:28.508373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:28.508344 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:28.638980 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:28.638828 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz"] Aug 05 12:36:28.641060 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:36:28.641035 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85b5332_1aa7_46b0_8f3c_159a6cf1d534.slice/crio-6dc386738f8d5aaadabc6884bdf8f9576085fabc0cd1f5a890654ee41d720d29 WatchSource:0}: Error finding container 6dc386738f8d5aaadabc6884bdf8f9576085fabc0cd1f5a890654ee41d720d29: Status 404 returned error can't find the container with id 6dc386738f8d5aaadabc6884bdf8f9576085fabc0cd1f5a890654ee41d720d29 Aug 05 12:36:29.233571 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:29.233536 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerStarted","Data":"adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2"} Aug 05 12:36:29.233848 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:29.233592 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerStarted","Data":"6dc386738f8d5aaadabc6884bdf8f9576085fabc0cd1f5a890654ee41d720d29"} Aug 05 12:36:30.239664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:30.239624 2569 generic.go:356] "Generic (PLEG): container finished" podID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerID="adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2" exitCode=0 Aug 05 12:36:30.240068 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:30.239698 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerDied","Data":"adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2"} Aug 05 12:36:31.246429 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:31.246394 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerStarted","Data":"2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7"} Aug 05 12:36:37.275089 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:37.275058 2569 generic.go:356] "Generic (PLEG): container finished" podID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerID="2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7" exitCode=0 Aug 05 12:36:37.275423 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:37.275125 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerDied","Data":"2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7"} Aug 05 12:36:38.283246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:38.283206 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerStarted","Data":"4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75"} Aug 05 12:36:38.283754 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:38.283291 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerStarted","Data":"e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750"} Aug 05 12:36:38.305704 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:38.305647 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podStartSLOduration=11.305632514 podStartE2EDuration="11.305632514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:36:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:36:38.304181353 +0000 UTC m=+1063.369773253" watchObservedRunningTime="2026-08-05 12:36:38.305632514 +0000 UTC m=+1063.371224411" Aug 05 12:36:38.509235 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:38.509198 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:38.509400 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:38.509277 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:40.509493 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:40.509451 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:40.510021 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:40.509522 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:40.511126 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:40.511091 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused" Aug 05 12:36:40.511254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:40.511131 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Aug 05 12:36:43.509243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:43.509199 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused" Aug 05 12:36:43.514774 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:43.514745 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:43.518149 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:43.518125 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:46.508823 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:46.508780 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.38:6571: connect: connection refused" Aug 05 12:36:49.509572 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:49.509500 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:49.510338 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:49.510318 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:36:49.744245 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:49.744211 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:53.363315 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.363280 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:53.443300 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.443260 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:53.448123 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.448095 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv"] Aug 05 12:36:53.450714 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.450693 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.452799 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.452776 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Aug 05 12:36:53.452902 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.452779 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Aug 05 12:36:53.453083 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.453067 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Aug 05 12:36:53.453232 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.453214 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-nfx56\"" Aug 05 12:36:53.460759 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.460740 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv"] Aug 05 12:36:53.543578 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.543555 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/93908e4a-1d47-4d22-a99c-c4f7e8234548-feast-data\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.543688 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.543596 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.543688 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.543615 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qssl9\" (UniqueName: \"kubernetes.io/projected/93908e4a-1d47-4d22-a99c-c4f7e8234548-kube-api-access-qssl9\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.543813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.543684 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.644448 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.644379 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/93908e4a-1d47-4d22-a99c-c4f7e8234548-feast-data\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.644448 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.644428 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.644607 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:36:53.644569 2569 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Aug 05 12:36:53.644673 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:36:53.644622 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls podName:93908e4a-1d47-4d22-a99c-c4f7e8234548 nodeName:}" failed. No retries permitted until 2026-08-05 12:36:54.144607405 +0000 UTC m=+1079.210199281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls") pod "feast-driver-ranking-6499cdcb4b-4z7lv" (UID: "93908e4a-1d47-4d22-a99c-c4f7e8234548") : secret "feast-driver-ranking-online-tls" not found Aug 05 12:36:53.644673 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.644618 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qssl9\" (UniqueName: \"kubernetes.io/projected/93908e4a-1d47-4d22-a99c-c4f7e8234548-kube-api-access-qssl9\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.644673 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.644659 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.644818 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:36:53.644729 2569 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found Aug 05 12:36:53.644818 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.644725 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/93908e4a-1d47-4d22-a99c-c4f7e8234548-feast-data\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.644818 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:36:53.644751 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls podName:93908e4a-1d47-4d22-a99c-c4f7e8234548 nodeName:}" failed. No retries permitted until 2026-08-05 12:36:54.144744606 +0000 UTC m=+1079.210336482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls") pod "feast-driver-ranking-6499cdcb4b-4z7lv" (UID: "93908e4a-1d47-4d22-a99c-c4f7e8234548") : secret "feast-driver-ranking-registry-tls" not found Aug 05 12:36:53.652860 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.652832 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qssl9\" (UniqueName: \"kubernetes.io/projected/93908e4a-1d47-4d22-a99c-c4f7e8234548-kube-api-access-qssl9\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:53.719152 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.719126 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:53.822062 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:53.822029 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:36:54.148875 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:54.148835 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:54.149082 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:54.148998 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:54.151373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:54.151350 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:54.151479 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:54.151352 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls\") pod \"feast-driver-ranking-6499cdcb4b-4z7lv\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:54.360161 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:54.360137 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:36:54.489557 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:54.489532 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv"] Aug 05 12:36:54.490832 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:36:54.490805 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93908e4a_1d47_4d22_a99c_c4f7e8234548.slice/crio-bb25f7501598b16f3a49336ae6468f468081621936e9f10b04ab460c9f12ec60 WatchSource:0}: Error finding container bb25f7501598b16f3a49336ae6468f468081621936e9f10b04ab460c9f12ec60: Status 404 returned error can't find the container with id bb25f7501598b16f3a49336ae6468f468081621936e9f10b04ab460c9f12ec60 Aug 05 12:36:55.369377 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:55.369343 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerStarted","Data":"cbd8e4bad5f6a0b967d89c4c6a328e0961c06921571426e7f72bd0651bf87f46"} Aug 05 12:36:55.369547 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:55.369389 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerStarted","Data":"bb25f7501598b16f3a49336ae6468f468081621936e9f10b04ab460c9f12ec60"} Aug 05 12:36:59.388969 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:59.388870 2569 generic.go:356] "Generic (PLEG): container finished" podID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerID="cbd8e4bad5f6a0b967d89c4c6a328e0961c06921571426e7f72bd0651bf87f46" exitCode=0 Aug 05 12:36:59.388969 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:36:59.388914 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerDied","Data":"cbd8e4bad5f6a0b967d89c4c6a328e0961c06921571426e7f72bd0651bf87f46"} Aug 05 12:37:00.396435 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:00.396402 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerStarted","Data":"32f4a8032d5603e490cf35d493820c4ac87ab8a59d1b1df7194f92d31dd075c8"} Aug 05 12:37:06.432546 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:06.432512 2569 generic.go:356] "Generic (PLEG): container finished" podID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerID="32f4a8032d5603e490cf35d493820c4ac87ab8a59d1b1df7194f92d31dd075c8" exitCode=0 Aug 05 12:37:06.433088 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:06.432591 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerDied","Data":"32f4a8032d5603e490cf35d493820c4ac87ab8a59d1b1df7194f92d31dd075c8"} Aug 05 12:37:06.435801 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:06.435783 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Aug 05 12:37:07.295672 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:07.295638 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:37:07.442103 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:07.442064 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerStarted","Data":"3e753a8826fc48e75970ed3d3232e5e189272c50605847f2f47ff3dd760fdf3e"} Aug 05 12:37:07.442568 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:07.442124 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerStarted","Data":"c4d867182aeb5d3909c183728c33dbc904fd7623ce2fe2057bb454e44dcef7b2"} Aug 05 12:37:07.465381 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:07.465324 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podStartSLOduration=14.465303674 podStartE2EDuration="14.465303674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:37:07.462791997 +0000 UTC m=+1092.528383905" watchObservedRunningTime="2026-08-05 12:37:07.465303674 +0000 UTC m=+1092.530895573" Aug 05 12:37:09.361099 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:09.361062 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:37:09.361583 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:09.361140 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:37:09.362875 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:09.362846 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused" Aug 05 12:37:09.363036 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:09.362888 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Aug 05 12:37:12.361021 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:12.360971 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused" Aug 05 12:37:12.688545 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:12.688472 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:37:12.689070 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:12.689047 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:37:12.692974 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:12.692929 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:37:14.360773 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:14.360735 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:37:15.360476 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:15.360429 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused" Aug 05 12:37:18.361577 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:18.361548 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:37:18.362248 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:18.362227 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:37:38.669922 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:37:38.669884 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:29.766046 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:29.766003 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:32.344379 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.344339 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:32.628347 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.628268 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj"] Aug 05 12:38:32.628544 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.628515 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" podUID="482edd92-5ce9-4017-b649-999f37de1131" containerName="manager" containerID="cri-o://f68e5e190c8c6853f301b73f654968b146d3e1d3e24dec434acae57453b7e341" gracePeriod=10 Aug 05 12:38:32.638510 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.638475 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:32.846112 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.846073 2569 generic.go:356] "Generic (PLEG): container finished" podID="482edd92-5ce9-4017-b649-999f37de1131" containerID="f68e5e190c8c6853f301b73f654968b146d3e1d3e24dec434acae57453b7e341" exitCode=0 Aug 05 12:38:32.846298 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.846216 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" event={"ID":"482edd92-5ce9-4017-b649-999f37de1131","Type":"ContainerDied","Data":"f68e5e190c8c6853f301b73f654968b146d3e1d3e24dec434acae57453b7e341"} Aug 05 12:38:32.880194 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.880120 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:32.893172 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.893147 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:38:32.932769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.932731 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz"] Aug 05 12:38:32.933931 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.933892 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" containerID="cri-o://4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75" gracePeriod=30 Aug 05 12:38:32.934775 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.934281 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv"] Aug 05 12:38:32.934775 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.934685 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="registry" containerID="cri-o://c4d867182aeb5d3909c183728c33dbc904fd7623ce2fe2057bb454e44dcef7b2" gracePeriod=30 Aug 05 12:38:32.935102 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.934904 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="online" containerID="cri-o://3e753a8826fc48e75970ed3d3232e5e189272c50605847f2f47ff3dd760fdf3e" gracePeriod=30 Aug 05 12:38:32.935282 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.935200 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="registry" containerID="cri-o://e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750" gracePeriod=30 Aug 05 12:38:32.947536 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.947510 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:32.982451 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.982425 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x4g5\" (UniqueName: \"kubernetes.io/projected/482edd92-5ce9-4017-b649-999f37de1131-kube-api-access-7x4g5\") pod \"482edd92-5ce9-4017-b649-999f37de1131\" (UID: \"482edd92-5ce9-4017-b649-999f37de1131\") " Aug 05 12:38:32.984483 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:32.984458 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/482edd92-5ce9-4017-b649-999f37de1131-kube-api-access-7x4g5" (OuterVolumeSpecName: "kube-api-access-7x4g5") pod "482edd92-5ce9-4017-b649-999f37de1131" (UID: "482edd92-5ce9-4017-b649-999f37de1131"). InnerVolumeSpecName "kube-api-access-7x4g5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:38:33.083224 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.083180 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x4g5\" (UniqueName: \"kubernetes.io/projected/482edd92-5ce9-4017-b649-999f37de1131-kube-api-access-7x4g5\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:38:33.515439 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.515393 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Aug 05 12:38:33.851587 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.851562 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" Aug 05 12:38:33.851587 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.851570 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj" event={"ID":"482edd92-5ce9-4017-b649-999f37de1131","Type":"ContainerDied","Data":"1f69e547e279725f54f4a322119a3ea52d85266827e13f049412207511e67917"} Aug 05 12:38:33.851817 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.851635 2569 scope.go:117] "RemoveContainer" containerID="f68e5e190c8c6853f301b73f654968b146d3e1d3e24dec434acae57453b7e341" Aug 05 12:38:33.854412 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.854392 2569 generic.go:356] "Generic (PLEG): container finished" podID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerID="4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75" exitCode=0 Aug 05 12:38:33.854505 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.854454 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerDied","Data":"4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75"} Aug 05 12:38:33.856906 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.856884 2569 generic.go:356] "Generic (PLEG): container finished" podID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerID="3e753a8826fc48e75970ed3d3232e5e189272c50605847f2f47ff3dd760fdf3e" exitCode=0 Aug 05 12:38:33.857011 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.856941 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerDied","Data":"3e753a8826fc48e75970ed3d3232e5e189272c50605847f2f47ff3dd760fdf3e"} Aug 05 12:38:33.868334 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.868313 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj"] Aug 05 12:38:33.871321 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:33.871303 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-544888d49-fqcjj"] Aug 05 12:38:34.690436 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.690401 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nb2pm"] Aug 05 12:38:34.690842 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.690576 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" podUID="40a8a370-5e18-45e7-ad64-a21f197df3b0" containerName="postgres" containerID="cri-o://287ea3086d18411c0983d52d9eacf82d9f12d290ffed0b74565d90ef91d8fa1e" gracePeriod=30 Aug 05 12:38:34.695053 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.695029 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-wzwn4"] Aug 05 12:38:34.695198 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.695179 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" podUID="f2f98ee6-b4fd-40a5-915b-fe912ae38702" containerName="redis" containerID="cri-o://c76f4dbf728efe641f17e88eee99193039846279d4e40871cb7018ea5e3abe4e" gracePeriod=30 Aug 05 12:38:34.755304 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.755278 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:34.863106 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.863074 2569 generic.go:356] "Generic (PLEG): container finished" podID="f2f98ee6-b4fd-40a5-915b-fe912ae38702" containerID="c76f4dbf728efe641f17e88eee99193039846279d4e40871cb7018ea5e3abe4e" exitCode=0 Aug 05 12:38:34.863283 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.863148 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" event={"ID":"f2f98ee6-b4fd-40a5-915b-fe912ae38702","Type":"ContainerDied","Data":"c76f4dbf728efe641f17e88eee99193039846279d4e40871cb7018ea5e3abe4e"} Aug 05 12:38:34.866454 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.866298 2569 generic.go:356] "Generic (PLEG): container finished" podID="40a8a370-5e18-45e7-ad64-a21f197df3b0" containerID="287ea3086d18411c0983d52d9eacf82d9f12d290ffed0b74565d90ef91d8fa1e" exitCode=0 Aug 05 12:38:34.866454 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.866391 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" event={"ID":"40a8a370-5e18-45e7-ad64-a21f197df3b0","Type":"ContainerDied","Data":"287ea3086d18411c0983d52d9eacf82d9f12d290ffed0b74565d90ef91d8fa1e"} Aug 05 12:38:34.965590 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.965569 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" Aug 05 12:38:34.968263 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.968243 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:38:34.998891 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.998866 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9rb9\" (UniqueName: \"kubernetes.io/projected/f2f98ee6-b4fd-40a5-915b-fe912ae38702-kube-api-access-w9rb9\") pod \"f2f98ee6-b4fd-40a5-915b-fe912ae38702\" (UID: \"f2f98ee6-b4fd-40a5-915b-fe912ae38702\") " Aug 05 12:38:34.999050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.998914 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40a8a370-5e18-45e7-ad64-a21f197df3b0-postgresdata\") pod \"40a8a370-5e18-45e7-ad64-a21f197df3b0\" (UID: \"40a8a370-5e18-45e7-ad64-a21f197df3b0\") " Aug 05 12:38:34.999050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:34.999030 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2jrf\" (UniqueName: \"kubernetes.io/projected/40a8a370-5e18-45e7-ad64-a21f197df3b0-kube-api-access-m2jrf\") pod \"40a8a370-5e18-45e7-ad64-a21f197df3b0\" (UID: \"40a8a370-5e18-45e7-ad64-a21f197df3b0\") " Aug 05 12:38:35.001451 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.001411 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a8a370-5e18-45e7-ad64-a21f197df3b0-kube-api-access-m2jrf" (OuterVolumeSpecName: "kube-api-access-m2jrf") pod "40a8a370-5e18-45e7-ad64-a21f197df3b0" (UID: "40a8a370-5e18-45e7-ad64-a21f197df3b0"). InnerVolumeSpecName "kube-api-access-m2jrf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:38:35.001654 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.001624 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f98ee6-b4fd-40a5-915b-fe912ae38702-kube-api-access-w9rb9" (OuterVolumeSpecName: "kube-api-access-w9rb9") pod "f2f98ee6-b4fd-40a5-915b-fe912ae38702" (UID: "f2f98ee6-b4fd-40a5-915b-fe912ae38702"). InnerVolumeSpecName "kube-api-access-w9rb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:38:35.032634 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.032607 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a8a370-5e18-45e7-ad64-a21f197df3b0-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "40a8a370-5e18-45e7-ad64-a21f197df3b0" (UID: "40a8a370-5e18-45e7-ad64-a21f197df3b0"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:38:35.100216 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.100188 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2jrf\" (UniqueName: \"kubernetes.io/projected/40a8a370-5e18-45e7-ad64-a21f197df3b0-kube-api-access-m2jrf\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:38:35.100216 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.100215 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9rb9\" (UniqueName: \"kubernetes.io/projected/f2f98ee6-b4fd-40a5-915b-fe912ae38702-kube-api-access-w9rb9\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:38:35.100352 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.100229 2569 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40a8a370-5e18-45e7-ad64-a21f197df3b0-postgresdata\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:38:35.443655 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.443624 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="482edd92-5ce9-4017-b649-999f37de1131" path="/var/lib/kubelet/pods/482edd92-5ce9-4017-b649-999f37de1131/volumes" Aug 05 12:38:35.872414 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.872381 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" Aug 05 12:38:35.872824 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.872388 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-nb2pm" event={"ID":"40a8a370-5e18-45e7-ad64-a21f197df3b0","Type":"ContainerDied","Data":"8839c2d21fd0f6c5d8037d4be4956933e9eb392332b8d932011ab633c19a733f"} Aug 05 12:38:35.872824 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.872511 2569 scope.go:117] "RemoveContainer" containerID="287ea3086d18411c0983d52d9eacf82d9f12d290ffed0b74565d90ef91d8fa1e" Aug 05 12:38:35.873810 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.873750 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" event={"ID":"f2f98ee6-b4fd-40a5-915b-fe912ae38702","Type":"ContainerDied","Data":"3cbae61f8934b10299da290a711f95bed8745c4e000c01a15fa337e707860306"} Aug 05 12:38:35.873810 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.873759 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-wzwn4" Aug 05 12:38:35.880750 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.880724 2569 scope.go:117] "RemoveContainer" containerID="c76f4dbf728efe641f17e88eee99193039846279d4e40871cb7018ea5e3abe4e" Aug 05 12:38:35.889813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.889794 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nb2pm"] Aug 05 12:38:35.893487 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.893468 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-nb2pm"] Aug 05 12:38:35.902593 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.902571 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-wzwn4"] Aug 05 12:38:35.904419 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:35.904400 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-wzwn4"] Aug 05 12:38:37.441125 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:37.441089 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40a8a370-5e18-45e7-ad64-a21f197df3b0" path="/var/lib/kubelet/pods/40a8a370-5e18-45e7-ad64-a21f197df3b0/volumes" Aug 05 12:38:37.441493 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:37.441406 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f98ee6-b4fd-40a5-915b-fe912ae38702" path="/var/lib/kubelet/pods/f2f98ee6-b4fd-40a5-915b-fe912ae38702/volumes" Aug 05 12:38:37.835653 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:37.835623 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:37.892409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:37.892385 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:42.689645 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:42.689605 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Aug 05 12:38:43.269777 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:43.269742 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:43.369908 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:43.369877 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:43.515473 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:43.515436 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Aug 05 12:38:43.760602 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:43.760572 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:38:52.689893 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:52.689851 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Aug 05 12:38:53.516034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:53.515991 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Aug 05 12:38:53.516188 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:53.516110 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:38:55.377785 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:55.377755 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:38:55.381988 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:55.381946 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:38:59.988679 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:38:59.988626 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:00.097933 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:00.097904 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:02.689331 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:02.689293 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Aug 05 12:39:02.689806 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:02.689407 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:39:02.988938 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:02.988904 2569 generic.go:356] "Generic (PLEG): container finished" podID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerID="c4d867182aeb5d3909c183728c33dbc904fd7623ce2fe2057bb454e44dcef7b2" exitCode=137 Aug 05 12:39:02.989076 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:02.988992 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerDied","Data":"c4d867182aeb5d3909c183728c33dbc904fd7623ce2fe2057bb454e44dcef7b2"} Aug 05 12:39:03.515816 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.515782 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Aug 05 12:39:03.606886 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.606864 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:39:03.609466 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.609447 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:39:03.731802 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.731731 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls\") pod \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " Aug 05 12:39:03.731802 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.731781 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-feast-data\") pod \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " Aug 05 12:39:03.732326 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.731806 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/93908e4a-1d47-4d22-a99c-c4f7e8234548-feast-data\") pod \"93908e4a-1d47-4d22-a99c-c4f7e8234548\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " Aug 05 12:39:03.732326 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.731859 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrzs6\" (UniqueName: \"kubernetes.io/projected/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-kube-api-access-vrzs6\") pod \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " Aug 05 12:39:03.732326 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.731884 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-registry-tls\") pod \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\" (UID: \"e85b5332-1aa7-46b0-8f3c-159a6cf1d534\") " Aug 05 12:39:03.732326 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.731906 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls\") pod \"93908e4a-1d47-4d22-a99c-c4f7e8234548\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " Aug 05 12:39:03.732326 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.731982 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls\") pod \"93908e4a-1d47-4d22-a99c-c4f7e8234548\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " Aug 05 12:39:03.732326 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.732005 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qssl9\" (UniqueName: \"kubernetes.io/projected/93908e4a-1d47-4d22-a99c-c4f7e8234548-kube-api-access-qssl9\") pod \"93908e4a-1d47-4d22-a99c-c4f7e8234548\" (UID: \"93908e4a-1d47-4d22-a99c-c4f7e8234548\") " Aug 05 12:39:03.732645 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.732596 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93908e4a-1d47-4d22-a99c-c4f7e8234548-feast-data" (OuterVolumeSpecName: "feast-data") pod "93908e4a-1d47-4d22-a99c-c4f7e8234548" (UID: "93908e4a-1d47-4d22-a99c-c4f7e8234548"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:39:03.734346 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.734313 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e85b5332-1aa7-46b0-8f3c-159a6cf1d534" (UID: "e85b5332-1aa7-46b0-8f3c-159a6cf1d534"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:39:03.734462 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.734324 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls" (OuterVolumeSpecName: "online-tls") pod "e85b5332-1aa7-46b0-8f3c-159a6cf1d534" (UID: "e85b5332-1aa7-46b0-8f3c-159a6cf1d534"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:39:03.734462 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.734386 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls" (OuterVolumeSpecName: "online-tls") pod "93908e4a-1d47-4d22-a99c-c4f7e8234548" (UID: "93908e4a-1d47-4d22-a99c-c4f7e8234548"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:39:03.734462 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.734424 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-kube-api-access-vrzs6" (OuterVolumeSpecName: "kube-api-access-vrzs6") pod "e85b5332-1aa7-46b0-8f3c-159a6cf1d534" (UID: "e85b5332-1aa7-46b0-8f3c-159a6cf1d534"). InnerVolumeSpecName "kube-api-access-vrzs6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:39:03.734578 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.734493 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93908e4a-1d47-4d22-a99c-c4f7e8234548-kube-api-access-qssl9" (OuterVolumeSpecName: "kube-api-access-qssl9") pod "93908e4a-1d47-4d22-a99c-c4f7e8234548" (UID: "93908e4a-1d47-4d22-a99c-c4f7e8234548"). InnerVolumeSpecName "kube-api-access-qssl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:39:03.734938 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.734910 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "93908e4a-1d47-4d22-a99c-c4f7e8234548" (UID: "93908e4a-1d47-4d22-a99c-c4f7e8234548"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:39:03.741500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.741478 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-feast-data" (OuterVolumeSpecName: "feast-data") pod "e85b5332-1aa7-46b0-8f3c-159a6cf1d534" (UID: "e85b5332-1aa7-46b0-8f3c-159a6cf1d534"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:39:03.833212 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.833189 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:39:03.833212 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.833212 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qssl9\" (UniqueName: \"kubernetes.io/projected/93908e4a-1d47-4d22-a99c-c4f7e8234548-kube-api-access-qssl9\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:39:03.833370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.833224 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:39:03.833370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.833234 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:39:03.833370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.833242 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/93908e4a-1d47-4d22-a99c-c4f7e8234548-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:39:03.833370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.833250 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrzs6\" (UniqueName: \"kubernetes.io/projected/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-kube-api-access-vrzs6\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:39:03.833370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.833258 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e85b5332-1aa7-46b0-8f3c-159a6cf1d534-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:39:03.833370 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.833266 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/93908e4a-1d47-4d22-a99c-c4f7e8234548-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:39:03.995264 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.995191 2569 generic.go:356] "Generic (PLEG): container finished" podID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerID="e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750" exitCode=137 Aug 05 12:39:03.995264 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.995253 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" Aug 05 12:39:03.995438 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.995278 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerDied","Data":"e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750"} Aug 05 12:39:03.995438 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.995337 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz" event={"ID":"e85b5332-1aa7-46b0-8f3c-159a6cf1d534","Type":"ContainerDied","Data":"6dc386738f8d5aaadabc6884bdf8f9576085fabc0cd1f5a890654ee41d720d29"} Aug 05 12:39:03.995438 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:03.995367 2569 scope.go:117] "RemoveContainer" containerID="4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75" Aug 05 12:39:04.000578 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.000545 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" event={"ID":"93908e4a-1d47-4d22-a99c-c4f7e8234548","Type":"ContainerDied","Data":"bb25f7501598b16f3a49336ae6468f468081621936e9f10b04ab460c9f12ec60"} Aug 05 12:39:04.000705 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.000564 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv" Aug 05 12:39:04.006222 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.006200 2569 scope.go:117] "RemoveContainer" containerID="e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750" Aug 05 12:39:04.014793 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.014772 2569 scope.go:117] "RemoveContainer" containerID="2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7" Aug 05 12:39:04.022327 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.022309 2569 scope.go:117] "RemoveContainer" containerID="adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2" Aug 05 12:39:04.028152 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.028130 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz"] Aug 05 12:39:04.029717 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.029702 2569 scope.go:117] "RemoveContainer" containerID="4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75" Aug 05 12:39:04.029981 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:39:04.029961 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75\": container with ID starting with 4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75 not found: ID does not exist" containerID="4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75" Aug 05 12:39:04.030063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.029996 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75"} err="failed to get container status \"4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75\": rpc error: code = NotFound desc = could not find container \"4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75\": container with ID starting with 4513c544712b5afcd41af846d7a660d1c10ae8d6be56983cb12b1dbbdf092c75 not found: ID does not exist" Aug 05 12:39:04.030063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.030022 2569 scope.go:117] "RemoveContainer" containerID="e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750" Aug 05 12:39:04.030289 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:39:04.030272 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750\": container with ID starting with e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750 not found: ID does not exist" containerID="e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750" Aug 05 12:39:04.030331 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.030296 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750"} err="failed to get container status \"e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750\": rpc error: code = NotFound desc = could not find container \"e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750\": container with ID starting with e2512747b78bec115d264acb87c636d626db75e43976b56051ea00cb741be750 not found: ID does not exist" Aug 05 12:39:04.030331 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.030311 2569 scope.go:117] "RemoveContainer" containerID="2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7" Aug 05 12:39:04.030530 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:39:04.030513 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7\": container with ID starting with 2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7 not found: ID does not exist" containerID="2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7" Aug 05 12:39:04.030585 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.030537 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7"} err="failed to get container status \"2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7\": rpc error: code = NotFound desc = could not find container \"2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7\": container with ID starting with 2e792c41fbd99faaab2b2e092c19b3ff038d6c8a22fc3fc2d021380b673c8dc7 not found: ID does not exist" Aug 05 12:39:04.030585 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.030557 2569 scope.go:117] "RemoveContainer" containerID="adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2" Aug 05 12:39:04.030800 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:39:04.030784 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2\": container with ID starting with adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2 not found: ID does not exist" containerID="adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2" Aug 05 12:39:04.030838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.030804 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2"} err="failed to get container status \"adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2\": rpc error: code = NotFound desc = could not find container \"adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2\": container with ID starting with adf86590247172d27327bd27ba0f24c0c890fe73089795307bc840ae2ecc67f2 not found: ID does not exist" Aug 05 12:39:04.030838 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.030819 2569 scope.go:117] "RemoveContainer" containerID="3e753a8826fc48e75970ed3d3232e5e189272c50605847f2f47ff3dd760fdf3e" Aug 05 12:39:04.034389 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.034367 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-985d988fc-r8shz"] Aug 05 12:39:04.038295 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.038280 2569 scope.go:117] "RemoveContainer" containerID="c4d867182aeb5d3909c183728c33dbc904fd7623ce2fe2057bb454e44dcef7b2" Aug 05 12:39:04.045482 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.045458 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv"] Aug 05 12:39:04.045809 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.045784 2569 scope.go:117] "RemoveContainer" containerID="32f4a8032d5603e490cf35d493820c4ac87ab8a59d1b1df7194f92d31dd075c8" Aug 05 12:39:04.051169 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.051150 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6499cdcb4b-4z7lv"] Aug 05 12:39:04.053167 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:04.053153 2569 scope.go:117] "RemoveContainer" containerID="cbd8e4bad5f6a0b967d89c4c6a328e0961c06921571426e7f72bd0651bf87f46" Aug 05 12:39:05.441155 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.441121 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" path="/var/lib/kubelet/pods/93908e4a-1d47-4d22-a99c-c4f7e8234548/volumes" Aug 05 12:39:05.441646 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.441632 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" path="/var/lib/kubelet/pods/e85b5332-1aa7-46b0-8f3c-159a6cf1d534/volumes" Aug 05 12:39:05.751536 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.751453 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:05.854598 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.854565 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:05.863234 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863209 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x"] Aug 05 12:39:05.863563 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863549 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="feast-apply" Aug 05 12:39:05.863612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863565 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="feast-apply" Aug 05 12:39:05.863612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863580 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="registry" Aug 05 12:39:05.863612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863586 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="registry" Aug 05 12:39:05.863612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863592 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" Aug 05 12:39:05.863612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863598 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" Aug 05 12:39:05.863612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863603 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="feast-init" Aug 05 12:39:05.863612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863609 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="feast-init" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863622 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="registry" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863627 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="registry" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863635 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482edd92-5ce9-4017-b649-999f37de1131" containerName="manager" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863640 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="482edd92-5ce9-4017-b649-999f37de1131" containerName="manager" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863647 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="feast-apply" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863652 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="feast-apply" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863658 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="online" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863662 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="online" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863672 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f98ee6-b4fd-40a5-915b-fe912ae38702" containerName="redis" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863677 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f98ee6-b4fd-40a5-915b-fe912ae38702" containerName="redis" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863682 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="feast-init" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863687 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="feast-init" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863695 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a8a370-5e18-45e7-ad64-a21f197df3b0" containerName="postgres" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863700 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a8a370-5e18-45e7-ad64-a21f197df3b0" containerName="postgres" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863749 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="40a8a370-5e18-45e7-ad64-a21f197df3b0" containerName="postgres" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863756 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="online" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863762 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="482edd92-5ce9-4017-b649-999f37de1131" containerName="manager" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863767 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="registry" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863775 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="93908e4a-1d47-4d22-a99c-c4f7e8234548" containerName="registry" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863781 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e85b5332-1aa7-46b0-8f3c-159a6cf1d534" containerName="online" Aug 05 12:39:05.863832 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.863786 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f98ee6-b4fd-40a5-915b-fe912ae38702" containerName="redis" Aug 05 12:39:05.869397 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.869381 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:05.872092 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.872069 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Aug 05 12:39:05.872227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.872099 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Aug 05 12:39:05.872227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.872126 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Aug 05 12:39:05.872348 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.872238 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Aug 05 12:39:05.872348 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.872240 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-97gqb\"" Aug 05 12:39:05.872597 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.872581 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Aug 05 12:39:05.872667 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.872582 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Aug 05 12:39:05.881573 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.881552 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x"] Aug 05 12:39:05.915543 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.915504 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:05.947610 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.947588 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:05.947736 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.947640 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:05.947736 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.947723 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef25dfb3-b582-4616-a115-b42a3f4bd488-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:05.947820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.947752 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:05.947820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.947794 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9hcj\" (UniqueName: \"kubernetes.io/projected/ef25dfb3-b582-4616-a115-b42a3f4bd488-kube-api-access-w9hcj\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:05.947898 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:05.947857 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.049097 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.049068 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.049250 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.049115 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.049250 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.049148 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.049250 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.049190 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef25dfb3-b582-4616-a115-b42a3f4bd488-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.049250 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.049215 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.049483 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:39:06.049285 2569 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Aug 05 12:39:06.049483 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:39:06.049343 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls podName:ef25dfb3-b582-4616-a115-b42a3f4bd488 nodeName:}" failed. No retries permitted until 2026-08-05 12:39:06.549322363 +0000 UTC m=+1211.614914240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls") pod "feast-simple-feast-setup-777d6cc998-l4g8x" (UID: "ef25dfb3-b582-4616-a115-b42a3f4bd488") : secret "feast-simple-feast-setup-offline-tls" not found Aug 05 12:39:06.049483 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.049358 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9hcj\" (UniqueName: \"kubernetes.io/projected/ef25dfb3-b582-4616-a115-b42a3f4bd488-kube-api-access-w9hcj\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.049650 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.049622 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef25dfb3-b582-4616-a115-b42a3f4bd488-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.051627 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.051600 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.051747 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.051685 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.051747 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.051741 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.059045 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.059014 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9hcj\" (UniqueName: \"kubernetes.io/projected/ef25dfb3-b582-4616-a115-b42a3f4bd488-kube-api-access-w9hcj\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.199580 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.199552 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:06.262301 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.262270 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:06.402013 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.401925 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:06.553241 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.553213 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.555474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.555452 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-l4g8x\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.787977 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.787937 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:06.922032 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.922007 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x"] Aug 05 12:39:06.923549 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:39:06.923522 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef25dfb3_b582_4616_a115_b42a3f4bd488.slice/crio-f37b4f76492963c209f9cd95872399c4aa6efbe52ecada6d40ecaac2969fc540 WatchSource:0}: Error finding container f37b4f76492963c209f9cd95872399c4aa6efbe52ecada6d40ecaac2969fc540: Status 404 returned error can't find the container with id f37b4f76492963c209f9cd95872399c4aa6efbe52ecada6d40ecaac2969fc540 Aug 05 12:39:06.925459 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:06.925441 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 12:39:07.015361 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:07.015336 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerStarted","Data":"f37b4f76492963c209f9cd95872399c4aa6efbe52ecada6d40ecaac2969fc540"} Aug 05 12:39:12.329196 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:12.329155 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:12.417658 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:12.417612 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:12.768474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:12.768439 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:19.089848 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:19.089813 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:39:20.086459 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:20.086420 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerStarted","Data":"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92"} Aug 05 12:39:25.108191 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:25.108154 2569 generic.go:356] "Generic (PLEG): container finished" podID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerID="5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92" exitCode=0 Aug 05 12:39:25.108594 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:25.108226 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerDied","Data":"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92"} Aug 05 12:39:26.114463 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:26.114425 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerStarted","Data":"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40"} Aug 05 12:39:31.135511 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:31.135479 2569 generic.go:356] "Generic (PLEG): container finished" podID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerID="1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40" exitCode=0 Aug 05 12:39:31.135984 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:31.135559 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerDied","Data":"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40"} Aug 05 12:39:32.153988 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:32.150609 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerStarted","Data":"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b"} Aug 05 12:39:32.153988 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:32.150688 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerStarted","Data":"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636"} Aug 05 12:39:32.153988 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:32.150711 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerStarted","Data":"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512"} Aug 05 12:39:32.153988 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:32.150736 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerStarted","Data":"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032"} Aug 05 12:39:32.181987 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:32.181910 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podStartSLOduration=14.542106268 podStartE2EDuration="27.181888874s" totalImagesPullingTime="12.639782606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:39:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:39:32.177894081 +0000 UTC m=+1237.243486014" watchObservedRunningTime="2026-08-05 12:39:32.181888874 +0000 UTC m=+1237.247480773" Aug 05 12:39:33.788732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:33.788637 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:33.788732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:33.788700 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:33.788732 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:33.788720 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:33.789966 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:33.789612 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:33.790935 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:33.790803 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Aug 05 12:39:33.790935 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:33.790839 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.40:8816: connect: connection refused" Aug 05 12:39:33.790935 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:33.790920 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.40:6571: connect: connection refused" Aug 05 12:39:33.791223 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:33.791053 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Aug 05 12:39:36.788546 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:36.788502 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:36.789078 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:36.788566 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:36.789078 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:36.788583 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:36.789078 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:36.788599 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:36.789078 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:36.788749 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Aug 05 12:39:36.789078 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:36.788866 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Aug 05 12:39:36.789403 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:36.789240 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.40:6571: connect: connection refused" Aug 05 12:39:36.790131 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:36.790106 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:37.180477 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:37.180397 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:39.788835 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:39.788802 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:39.789270 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:39.789031 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:39.789729 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:39.789705 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:39.789990 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:39.789973 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:39.794365 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:39.794345 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:39:40.194639 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:39:40.194569 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:40:41.552507 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.552477 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:41.554509 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.554480 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x"] Aug 05 12:40:41.554764 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.554740 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" containerID="cri-o://3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b" gracePeriod=30 Aug 05 12:40:41.554884 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.554798 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" containerID="cri-o://3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512" gracePeriod=30 Aug 05 12:40:41.554884 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.554728 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="registry" containerID="cri-o://fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032" gracePeriod=30 Aug 05 12:40:41.555192 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.555148 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="offline" containerID="cri-o://68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636" gracePeriod=30 Aug 05 12:40:41.607377 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.607346 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:41.746142 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.746109 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:41.845748 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.845659 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:41.866519 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.866485 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w"] Aug 05 12:40:41.870474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.870453 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:41.872624 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.872601 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-84xkk\"" Aug 05 12:40:41.886439 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.886416 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w"] Aug 05 12:40:41.931616 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.931575 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:41.931616 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.931612 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad7bda7-2959-4a5f-935c-8ee71d561674-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:41.931821 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.931661 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:41.931821 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.931701 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:41.931821 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.931747 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:41.931923 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.931834 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhk4w\" (UniqueName: \"kubernetes.io/projected/6ad7bda7-2959-4a5f-935c-8ee71d561674-kube-api-access-vhk4w\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:41.947432 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:41.947404 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:42.032825 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.032796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhk4w\" (UniqueName: \"kubernetes.io/projected/6ad7bda7-2959-4a5f-935c-8ee71d561674-kube-api-access-vhk4w\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.032991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.032842 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.032991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.032874 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad7bda7-2959-4a5f-935c-8ee71d561674-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.032991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.032937 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.032991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.032987 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.033229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.033014 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.033229 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:40:42.033152 2569 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Aug 05 12:40:42.033229 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:40:42.033226 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls podName:6ad7bda7-2959-4a5f-935c-8ee71d561674 nodeName:}" failed. No retries permitted until 2026-08-05 12:40:42.533205446 +0000 UTC m=+1307.598797326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls") pod "feast-simple-feast-setup-777d6cc998-mkg5w" (UID: "6ad7bda7-2959-4a5f-935c-8ee71d561674") : secret "feast-simple-feast-setup-ui-tls" not found Aug 05 12:40:42.033559 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.033513 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad7bda7-2959-4a5f-935c-8ee71d561674-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.035794 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.035757 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.035990 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.035969 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.035990 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.035982 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.042991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.042968 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhk4w\" (UniqueName: \"kubernetes.io/projected/6ad7bda7-2959-4a5f-935c-8ee71d561674-kube-api-access-vhk4w\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.113801 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.113681 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:42.165931 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.165898 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:42.444146 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.444071 2569 generic.go:356] "Generic (PLEG): container finished" podID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerID="3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b" exitCode=0 Aug 05 12:40:42.444146 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.444096 2569 generic.go:356] "Generic (PLEG): container finished" podID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerID="3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512" exitCode=0 Aug 05 12:40:42.444146 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.444138 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerDied","Data":"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b"} Aug 05 12:40:42.444337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.444176 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerDied","Data":"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512"} Aug 05 12:40:42.538716 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.538691 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.541005 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.540977 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-mkg5w\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.780133 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.780112 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:42.909758 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:42.909730 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w"] Aug 05 12:40:42.911493 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:40:42.911469 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ad7bda7_2959_4a5f_935c_8ee71d561674.slice/crio-eba9949dce31b66c89c10393ce0b0b1ab63bfb87e8eab8b1d49fc47c25319eac WatchSource:0}: Error finding container eba9949dce31b66c89c10393ce0b0b1ab63bfb87e8eab8b1d49fc47c25319eac: Status 404 returned error can't find the container with id eba9949dce31b66c89c10393ce0b0b1ab63bfb87e8eab8b1d49fc47c25319eac Aug 05 12:40:43.450382 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:43.450348 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerStarted","Data":"1abf3008a3f1b1a11353bfaf8430d93a4a8c045732d13f9d48d41db69388af02"} Aug 05 12:40:43.450562 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:43.450394 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerStarted","Data":"eba9949dce31b66c89c10393ce0b0b1ab63bfb87e8eab8b1d49fc47c25319eac"} Aug 05 12:40:45.905680 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:45.905629 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:46.151090 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:46.151053 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:40:48.472425 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:48.472373 2569 generic.go:356] "Generic (PLEG): container finished" podID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerID="1abf3008a3f1b1a11353bfaf8430d93a4a8c045732d13f9d48d41db69388af02" exitCode=0 Aug 05 12:40:48.472868 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:48.472445 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerDied","Data":"1abf3008a3f1b1a11353bfaf8430d93a4a8c045732d13f9d48d41db69388af02"} Aug 05 12:40:49.477867 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:49.477829 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerStarted","Data":"980ccba099346ffcf40f45f8ecfc4ff20d4b79731bcd8ca35e0cfe7c69d733a3"} Aug 05 12:40:49.789965 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:49.789908 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Aug 05 12:40:50.191878 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:50.191794 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Aug 05 12:40:54.499803 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:54.499771 2569 generic.go:356] "Generic (PLEG): container finished" podID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerID="980ccba099346ffcf40f45f8ecfc4ff20d4b79731bcd8ca35e0cfe7c69d733a3" exitCode=0 Aug 05 12:40:54.500248 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:54.499841 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerDied","Data":"980ccba099346ffcf40f45f8ecfc4ff20d4b79731bcd8ca35e0cfe7c69d733a3"} Aug 05 12:40:55.510102 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:55.510057 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerStarted","Data":"258fda06fac5eae29463241f5dcebc253faf959b6624c9b6879e7a6f50f2778d"} Aug 05 12:40:55.510581 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:55.510124 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerStarted","Data":"b3b192f80fe5ac94ddbe6364b3cfe2c4bafd0963bd75baf74e51504987c60004"} Aug 05 12:40:55.510581 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:55.510146 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerStarted","Data":"abbb872534d694d09bf7e4c7ecacf776490c02f2b070070366a5aadf497b64ad"} Aug 05 12:40:55.510581 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:55.510167 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerStarted","Data":"3eeb2aaeb75486f1ecba640f67e785288fab17c843858f7a1542c49fa4c295da"} Aug 05 12:40:55.536390 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:55.536327 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podStartSLOduration=14.536306053 podStartE2EDuration="14.536306053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:40:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:40:55.533339765 +0000 UTC m=+1320.598931687" watchObservedRunningTime="2026-08-05 12:40:55.536306053 +0000 UTC m=+1320.601897955" Aug 05 12:40:57.780806 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:57.780773 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:57.781377 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:57.781360 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:57.781489 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:57.781478 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:57.781858 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:57.781837 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:40:57.783069 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:57.783036 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Aug 05 12:40:57.783373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:57.783063 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.41:8816: connect: connection refused" Aug 05 12:40:57.783373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:57.783152 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Aug 05 12:40:57.783373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:57.783088 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.41:6571: connect: connection refused" Aug 05 12:40:59.790309 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:40:59.790255 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Aug 05 12:41:00.192610 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:00.192497 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Aug 05 12:41:00.781800 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:00.781767 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:00.782359 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:00.782335 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:00.782497 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:00.782371 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:00.782497 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:00.782441 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:00.782578 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:00.782545 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:00.787449 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:00.787430 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:01.544558 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:01.544523 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:01.545061 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:01.544613 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:01.545061 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:01.544635 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:01.545061 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:01.544996 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:01.545199 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:01.545159 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:01.547991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:01.547976 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:41:09.789774 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:09.789730 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Aug 05 12:41:09.790309 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:09.789870 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:41:10.192284 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:10.192190 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Aug 05 12:41:10.192451 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:10.192333 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:41:12.198416 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.198385 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:41:12.202793 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.202768 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls\") pod \"ef25dfb3-b582-4616-a115-b42a3f4bd488\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " Aug 05 12:41:12.202930 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.202912 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9hcj\" (UniqueName: \"kubernetes.io/projected/ef25dfb3-b582-4616-a115-b42a3f4bd488-kube-api-access-w9hcj\") pod \"ef25dfb3-b582-4616-a115-b42a3f4bd488\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " Aug 05 12:41:12.202994 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.202984 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-registry-tls\") pod \"ef25dfb3-b582-4616-a115-b42a3f4bd488\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " Aug 05 12:41:12.203039 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.203009 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-online-tls\") pod \"ef25dfb3-b582-4616-a115-b42a3f4bd488\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " Aug 05 12:41:12.203076 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.203046 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-ui-tls\") pod \"ef25dfb3-b582-4616-a115-b42a3f4bd488\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " Aug 05 12:41:12.203128 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.203086 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef25dfb3-b582-4616-a115-b42a3f4bd488-feast-data\") pod \"ef25dfb3-b582-4616-a115-b42a3f4bd488\" (UID: \"ef25dfb3-b582-4616-a115-b42a3f4bd488\") " Aug 05 12:41:12.204027 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.204000 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef25dfb3-b582-4616-a115-b42a3f4bd488-feast-data" (OuterVolumeSpecName: "feast-data") pod "ef25dfb3-b582-4616-a115-b42a3f4bd488" (UID: "ef25dfb3-b582-4616-a115-b42a3f4bd488"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:41:12.205072 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.205051 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ef25dfb3-b582-4616-a115-b42a3f4bd488" (UID: "ef25dfb3-b582-4616-a115-b42a3f4bd488"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:41:12.205431 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.205408 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-online-tls" (OuterVolumeSpecName: "online-tls") pod "ef25dfb3-b582-4616-a115-b42a3f4bd488" (UID: "ef25dfb3-b582-4616-a115-b42a3f4bd488"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:41:12.205510 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.205454 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ef25dfb3-b582-4616-a115-b42a3f4bd488" (UID: "ef25dfb3-b582-4616-a115-b42a3f4bd488"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:41:12.205510 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.205484 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef25dfb3-b582-4616-a115-b42a3f4bd488-kube-api-access-w9hcj" (OuterVolumeSpecName: "kube-api-access-w9hcj") pod "ef25dfb3-b582-4616-a115-b42a3f4bd488" (UID: "ef25dfb3-b582-4616-a115-b42a3f4bd488"). InnerVolumeSpecName "kube-api-access-w9hcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:41:12.205633 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.205507 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ef25dfb3-b582-4616-a115-b42a3f4bd488" (UID: "ef25dfb3-b582-4616-a115-b42a3f4bd488"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:41:12.303855 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.303826 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:41:12.303855 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.303856 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9hcj\" (UniqueName: \"kubernetes.io/projected/ef25dfb3-b582-4616-a115-b42a3f4bd488-kube-api-access-w9hcj\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:41:12.304050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.303871 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:41:12.304050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.303883 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:41:12.304050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.303894 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef25dfb3-b582-4616-a115-b42a3f4bd488-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:41:12.304050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.303904 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef25dfb3-b582-4616-a115-b42a3f4bd488-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:41:12.597212 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.597180 2569 generic.go:356] "Generic (PLEG): container finished" podID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerID="68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636" exitCode=137 Aug 05 12:41:12.597212 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.597205 2569 generic.go:356] "Generic (PLEG): container finished" podID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerID="fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032" exitCode=137 Aug 05 12:41:12.597402 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.597237 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerDied","Data":"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636"} Aug 05 12:41:12.597402 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.597253 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" Aug 05 12:41:12.597402 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.597281 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerDied","Data":"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032"} Aug 05 12:41:12.597402 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.597296 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x" event={"ID":"ef25dfb3-b582-4616-a115-b42a3f4bd488","Type":"ContainerDied","Data":"f37b4f76492963c209f9cd95872399c4aa6efbe52ecada6d40ecaac2969fc540"} Aug 05 12:41:12.597402 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.597313 2569 scope.go:117] "RemoveContainer" containerID="3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b" Aug 05 12:41:12.606805 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.606789 2569 scope.go:117] "RemoveContainer" containerID="68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636" Aug 05 12:41:12.614427 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.614411 2569 scope.go:117] "RemoveContainer" containerID="3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512" Aug 05 12:41:12.620080 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.620061 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x"] Aug 05 12:41:12.622111 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.622097 2569 scope.go:117] "RemoveContainer" containerID="fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032" Aug 05 12:41:12.626223 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.626196 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-l4g8x"] Aug 05 12:41:12.631797 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.631766 2569 scope.go:117] "RemoveContainer" containerID="1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40" Aug 05 12:41:12.639245 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.639209 2569 scope.go:117] "RemoveContainer" containerID="5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92" Aug 05 12:41:12.646243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.646227 2569 scope.go:117] "RemoveContainer" containerID="3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b" Aug 05 12:41:12.646505 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:41:12.646486 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b\": container with ID starting with 3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b not found: ID does not exist" containerID="3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b" Aug 05 12:41:12.646568 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.646513 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b"} err="failed to get container status \"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b\": rpc error: code = NotFound desc = could not find container \"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b\": container with ID starting with 3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b not found: ID does not exist" Aug 05 12:41:12.646568 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.646533 2569 scope.go:117] "RemoveContainer" containerID="68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636" Aug 05 12:41:12.646778 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:41:12.646762 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636\": container with ID starting with 68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636 not found: ID does not exist" containerID="68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636" Aug 05 12:41:12.646828 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.646783 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636"} err="failed to get container status \"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636\": rpc error: code = NotFound desc = could not find container \"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636\": container with ID starting with 68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636 not found: ID does not exist" Aug 05 12:41:12.646828 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.646804 2569 scope.go:117] "RemoveContainer" containerID="3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512" Aug 05 12:41:12.647070 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:41:12.647053 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512\": container with ID starting with 3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512 not found: ID does not exist" containerID="3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512" Aug 05 12:41:12.647133 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647074 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512"} err="failed to get container status \"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512\": rpc error: code = NotFound desc = could not find container \"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512\": container with ID starting with 3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512 not found: ID does not exist" Aug 05 12:41:12.647133 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647089 2569 scope.go:117] "RemoveContainer" containerID="fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032" Aug 05 12:41:12.647308 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:41:12.647292 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032\": container with ID starting with fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032 not found: ID does not exist" containerID="fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032" Aug 05 12:41:12.647347 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647312 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032"} err="failed to get container status \"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032\": rpc error: code = NotFound desc = could not find container \"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032\": container with ID starting with fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032 not found: ID does not exist" Aug 05 12:41:12.647347 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647326 2569 scope.go:117] "RemoveContainer" containerID="1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40" Aug 05 12:41:12.647553 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:41:12.647539 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40\": container with ID starting with 1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40 not found: ID does not exist" containerID="1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40" Aug 05 12:41:12.647597 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647556 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40"} err="failed to get container status \"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40\": rpc error: code = NotFound desc = could not find container \"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40\": container with ID starting with 1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40 not found: ID does not exist" Aug 05 12:41:12.647597 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647568 2569 scope.go:117] "RemoveContainer" containerID="5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92" Aug 05 12:41:12.647762 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:41:12.647746 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92\": container with ID starting with 5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92 not found: ID does not exist" containerID="5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92" Aug 05 12:41:12.647804 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647766 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92"} err="failed to get container status \"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92\": rpc error: code = NotFound desc = could not find container \"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92\": container with ID starting with 5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92 not found: ID does not exist" Aug 05 12:41:12.647804 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647779 2569 scope.go:117] "RemoveContainer" containerID="3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b" Aug 05 12:41:12.647972 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647943 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b"} err="failed to get container status \"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b\": rpc error: code = NotFound desc = could not find container \"3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b\": container with ID starting with 3971e01d4f93e4e068b2ed6b31bcda8a5996faf33cf33ff19f7681b8a4bed58b not found: ID does not exist" Aug 05 12:41:12.648040 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.647972 2569 scope.go:117] "RemoveContainer" containerID="68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636" Aug 05 12:41:12.648191 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.648170 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636"} err="failed to get container status \"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636\": rpc error: code = NotFound desc = could not find container \"68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636\": container with ID starting with 68eee95e348aab679648260d69268267e7e6be8056fa811f1e222b33eb467636 not found: ID does not exist" Aug 05 12:41:12.648271 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.648193 2569 scope.go:117] "RemoveContainer" containerID="3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512" Aug 05 12:41:12.648437 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.648415 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512"} err="failed to get container status \"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512\": rpc error: code = NotFound desc = could not find container \"3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512\": container with ID starting with 3991efa2ff92ada0358155d4130ae9bd2e1527632c376bc2554784c60aa45512 not found: ID does not exist" Aug 05 12:41:12.648478 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.648441 2569 scope.go:117] "RemoveContainer" containerID="fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032" Aug 05 12:41:12.648668 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.648639 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032"} err="failed to get container status \"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032\": rpc error: code = NotFound desc = could not find container \"fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032\": container with ID starting with fedc9445e75bf0a4942dbdb7e0662f51cbf77a9fe58e9734600f5818db775032 not found: ID does not exist" Aug 05 12:41:12.648717 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.648671 2569 scope.go:117] "RemoveContainer" containerID="1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40" Aug 05 12:41:12.648915 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.648897 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40"} err="failed to get container status \"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40\": rpc error: code = NotFound desc = could not find container \"1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40\": container with ID starting with 1c10f82665107fc4641ee71f107f5a1878a4c5096c982e8f3a6846c6db3f4a40 not found: ID does not exist" Aug 05 12:41:12.649009 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.648917 2569 scope.go:117] "RemoveContainer" containerID="5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92" Aug 05 12:41:12.649186 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:12.649168 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92"} err="failed to get container status \"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92\": rpc error: code = NotFound desc = could not find container \"5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92\": container with ID starting with 5abd8b3a3c383c3c9056e0281bbf19594dd1f00f64c0cb33abc15f0bfd2f6b92 not found: ID does not exist" Aug 05 12:41:13.441981 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:41:13.441931 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" path="/var/lib/kubelet/pods/ef25dfb3-b582-4616-a115-b42a3f4bd488/volumes" Aug 05 12:42:03.254608 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.254573 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:42:03.629518 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629482 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v"] Aug 05 12:42:03.629811 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629799 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="feast-init" Aug 05 12:42:03.629869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629812 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="feast-init" Aug 05 12:42:03.629869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629823 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="registry" Aug 05 12:42:03.629869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629829 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="registry" Aug 05 12:42:03.629869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629840 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="offline" Aug 05 12:42:03.629869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629846 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="offline" Aug 05 12:42:03.629869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629857 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" Aug 05 12:42:03.629869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629864 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" Aug 05 12:42:03.629869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629871 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="feast-apply" Aug 05 12:42:03.630166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629877 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="feast-apply" Aug 05 12:42:03.630166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629885 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" Aug 05 12:42:03.630166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629891 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" Aug 05 12:42:03.630166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629936 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="ui" Aug 05 12:42:03.630166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629944 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="registry" Aug 05 12:42:03.630166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629965 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="online" Aug 05 12:42:03.630166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.629978 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef25dfb3-b582-4616-a115-b42a3f4bd488" containerName="offline" Aug 05 12:42:03.634522 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.634504 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.636995 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.636963 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Aug 05 12:42:03.637119 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.637074 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Aug 05 12:42:03.637119 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.637086 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Aug 05 12:42:03.637243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.637123 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-c89n6\"" Aug 05 12:42:03.637243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.637088 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Aug 05 12:42:03.637243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.637088 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Aug 05 12:42:03.637243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.637199 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Aug 05 12:42:03.644930 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.644908 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v"] Aug 05 12:42:03.797793 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.797761 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/6e5f875a-0e37-4775-b4db-4bb00cb808d4-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.797793 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.797817 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.798091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.797848 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.798091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.797920 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2tmg\" (UniqueName: \"kubernetes.io/projected/6e5f875a-0e37-4775-b4db-4bb00cb808d4-kube-api-access-f2tmg\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.798091 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.798021 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e5f875a-0e37-4775-b4db-4bb00cb808d4-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.798345 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.798092 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.899251 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.899174 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2tmg\" (UniqueName: \"kubernetes.io/projected/6e5f875a-0e37-4775-b4db-4bb00cb808d4-kube-api-access-f2tmg\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.899251 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.899241 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e5f875a-0e37-4775-b4db-4bb00cb808d4-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.899460 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.899260 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.899460 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.899279 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/6e5f875a-0e37-4775-b4db-4bb00cb808d4-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.899460 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.899303 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.899460 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.899321 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.899460 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:42:03.899378 2569 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Aug 05 12:42:03.899460 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:42:03.899437 2569 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Aug 05 12:42:03.899719 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:42:03.899445 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls podName:6e5f875a-0e37-4775-b4db-4bb00cb808d4 nodeName:}" failed. No retries permitted until 2026-08-05 12:42:04.39942631 +0000 UTC m=+1389.465018188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls") pod "feast-simple-feast-remote-setup-db6955c6c-j9f9v" (UID: "6e5f875a-0e37-4775-b4db-4bb00cb808d4") : secret "feast-simple-feast-remote-setup-offline-tls" not found Aug 05 12:42:03.899719 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:42:03.899513 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls podName:6e5f875a-0e37-4775-b4db-4bb00cb808d4 nodeName:}" failed. No retries permitted until 2026-08-05 12:42:04.399495026 +0000 UTC m=+1389.465086916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls") pod "feast-simple-feast-remote-setup-db6955c6c-j9f9v" (UID: "6e5f875a-0e37-4775-b4db-4bb00cb808d4") : secret "feast-simple-feast-remote-setup-online-tls" not found Aug 05 12:42:03.899719 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.899649 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e5f875a-0e37-4775-b4db-4bb00cb808d4-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.900028 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.900009 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/6e5f875a-0e37-4775-b4db-4bb00cb808d4-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.901677 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.901650 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:03.907056 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:03.907035 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2tmg\" (UniqueName: \"kubernetes.io/projected/6e5f875a-0e37-4775-b4db-4bb00cb808d4-kube-api-access-f2tmg\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:04.403798 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:04.403754 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:04.404208 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:04.403811 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:04.406262 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:04.406240 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:04.406407 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:04.406386 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-j9f9v\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:04.543352 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:04.543323 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:04.678705 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:04.678673 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v"] Aug 05 12:42:04.680054 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:42:04.680032 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e5f875a_0e37_4775_b4db_4bb00cb808d4.slice/crio-b8cf5992c5eb45ec3a04132ed3bcfc640fa96be4c2d2a78ba7ece0ce85ca4d9d WatchSource:0}: Error finding container b8cf5992c5eb45ec3a04132ed3bcfc640fa96be4c2d2a78ba7ece0ce85ca4d9d: Status 404 returned error can't find the container with id b8cf5992c5eb45ec3a04132ed3bcfc640fa96be4c2d2a78ba7ece0ce85ca4d9d Aug 05 12:42:04.812790 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:04.812754 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerStarted","Data":"08b796209618aab424cd3849d436bdd6f08265d88f465b00f9ee51c661f60b62"} Aug 05 12:42:04.812936 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:04.812807 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerStarted","Data":"b8cf5992c5eb45ec3a04132ed3bcfc640fa96be4c2d2a78ba7ece0ce85ca4d9d"} Aug 05 12:42:05.584248 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:05.584192 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:42:09.837159 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:09.837128 2569 generic.go:356] "Generic (PLEG): container finished" podID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerID="08b796209618aab424cd3849d436bdd6f08265d88f465b00f9ee51c661f60b62" exitCode=0 Aug 05 12:42:09.837557 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:09.837175 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerDied","Data":"08b796209618aab424cd3849d436bdd6f08265d88f465b00f9ee51c661f60b62"} Aug 05 12:42:10.842855 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:10.842824 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerStarted","Data":"454de08b6ead9410315eec1f19943e6bd7a0e815a48d64471d1e8440ef5e3a59"} Aug 05 12:42:15.863924 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:15.863848 2569 generic.go:356] "Generic (PLEG): container finished" podID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerID="454de08b6ead9410315eec1f19943e6bd7a0e815a48d64471d1e8440ef5e3a59" exitCode=0 Aug 05 12:42:15.864301 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:15.863922 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerDied","Data":"454de08b6ead9410315eec1f19943e6bd7a0e815a48d64471d1e8440ef5e3a59"} Aug 05 12:42:16.877096 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:16.877051 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerStarted","Data":"965d4233a13f0c45df3de9a4e7f7eb02007fed50fac7d9ac0a2fb483867e7eea"} Aug 05 12:42:16.877459 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:16.877115 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerStarted","Data":"9b233a7410a0b6268c358359c45d4cabb4cfe8c6e176a4397b17b8848e115157"} Aug 05 12:42:16.877459 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:16.877135 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerStarted","Data":"c45d36dcf33813d4118364f0ae20c4dedd733bacddc21997b6d47818be79b522"} Aug 05 12:42:16.901584 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:16.901522 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podStartSLOduration=13.901502749 podStartE2EDuration="13.901502749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:42:16.898687648 +0000 UTC m=+1401.964279550" watchObservedRunningTime="2026-08-05 12:42:16.901502749 +0000 UTC m=+1401.967094647" Aug 05 12:42:19.544375 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:19.544339 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:19.544967 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:19.544926 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:19.545104 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:19.544973 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:19.546252 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:19.546212 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Aug 05 12:42:19.546363 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:19.546250 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.42:8816: connect: connection refused" Aug 05 12:42:19.546363 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:19.546291 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Aug 05 12:42:22.545197 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.545170 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:22.545678 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.545301 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:22.545754 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.545724 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:22.545807 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.545777 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:22.545970 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.545937 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:22.546095 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.546077 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:22.551793 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.551772 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:22.907196 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.907115 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:42:22.911668 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:42:22.911643 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:43:23.565652 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:23.565612 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:43:25.011110 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.011075 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v"] Aug 05 12:43:25.011529 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.011349 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="online" containerID="cri-o://c45d36dcf33813d4118364f0ae20c4dedd733bacddc21997b6d47818be79b522" gracePeriod=30 Aug 05 12:43:25.011529 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.011417 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="offline" containerID="cri-o://9b233a7410a0b6268c358359c45d4cabb4cfe8c6e176a4397b17b8848e115157" gracePeriod=30 Aug 05 12:43:25.011529 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.011429 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="ui" containerID="cri-o://965d4233a13f0c45df3de9a4e7f7eb02007fed50fac7d9ac0a2fb483867e7eea" gracePeriod=30 Aug 05 12:43:25.012900 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.012885 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:43:25.077525 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.077494 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:43:25.164982 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.164938 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w"] Aug 05 12:43:25.165335 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.165260 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="registry" containerID="cri-o://3eeb2aaeb75486f1ecba640f67e785288fab17c843858f7a1542c49fa4c295da" gracePeriod=30 Aug 05 12:43:25.165335 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.165286 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="offline" containerID="cri-o://b3b192f80fe5ac94ddbe6364b3cfe2c4bafd0963bd75baf74e51504987c60004" gracePeriod=30 Aug 05 12:43:25.165335 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.165299 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="online" containerID="cri-o://abbb872534d694d09bf7e4c7ecacf776490c02f2b070070366a5aadf497b64ad" gracePeriod=30 Aug 05 12:43:25.165551 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.165308 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="ui" containerID="cri-o://258fda06fac5eae29463241f5dcebc253faf959b6624c9b6879e7a6f50f2778d" gracePeriod=30 Aug 05 12:43:25.167159 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.167130 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:43:25.220445 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.220406 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:43:25.274700 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:25.274622 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:43:26.184115 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:26.184040 2569 generic.go:356] "Generic (PLEG): container finished" podID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerID="965d4233a13f0c45df3de9a4e7f7eb02007fed50fac7d9ac0a2fb483867e7eea" exitCode=0 Aug 05 12:43:26.184115 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:26.184064 2569 generic.go:356] "Generic (PLEG): container finished" podID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerID="c45d36dcf33813d4118364f0ae20c4dedd733bacddc21997b6d47818be79b522" exitCode=0 Aug 05 12:43:26.184525 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:26.184109 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerDied","Data":"965d4233a13f0c45df3de9a4e7f7eb02007fed50fac7d9ac0a2fb483867e7eea"} Aug 05 12:43:26.184525 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:26.184150 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerDied","Data":"c45d36dcf33813d4118364f0ae20c4dedd733bacddc21997b6d47818be79b522"} Aug 05 12:43:26.187133 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:26.187115 2569 generic.go:356] "Generic (PLEG): container finished" podID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerID="258fda06fac5eae29463241f5dcebc253faf959b6624c9b6879e7a6f50f2778d" exitCode=0 Aug 05 12:43:26.187133 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:26.187130 2569 generic.go:356] "Generic (PLEG): container finished" podID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerID="abbb872534d694d09bf7e4c7ecacf776490c02f2b070070366a5aadf497b64ad" exitCode=0 Aug 05 12:43:26.187280 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:26.187151 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerDied","Data":"258fda06fac5eae29463241f5dcebc253faf959b6624c9b6879e7a6f50f2778d"} Aug 05 12:43:26.187280 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:26.187174 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerDied","Data":"abbb872534d694d09bf7e4c7ecacf776490c02f2b070070366a5aadf497b64ad"} Aug 05 12:43:30.349686 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:30.349649 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:43:30.782757 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:30.782723 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Aug 05 12:43:31.544869 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:31.544809 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Aug 05 12:43:32.546274 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:32.546233 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Aug 05 12:43:32.907329 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:32.907247 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Aug 05 12:43:40.782743 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:40.782703 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Aug 05 12:43:41.544913 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:41.544870 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Aug 05 12:43:42.546041 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:42.545994 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Aug 05 12:43:42.907721 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:42.907638 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Aug 05 12:43:50.782717 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:50.782672 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Aug 05 12:43:50.783264 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:50.782812 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:43:51.545225 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:51.545189 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Aug 05 12:43:51.545446 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:51.545308 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:43:52.546142 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:52.546100 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.42:8443: connect: connection refused" Aug 05 12:43:52.546610 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:52.546226 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:43:52.907803 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:52.907719 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="online" probeResult="failure" output="Get \"https://10.132.0.42:6567/health\": dial tcp 10.132.0.42:6567: connect: connection refused" Aug 05 12:43:52.907979 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:52.907853 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:43:55.310546 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.310468 2569 generic.go:356] "Generic (PLEG): container finished" podID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerID="b3b192f80fe5ac94ddbe6364b3cfe2c4bafd0963bd75baf74e51504987c60004" exitCode=137 Aug 05 12:43:55.310546 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.310491 2569 generic.go:356] "Generic (PLEG): container finished" podID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerID="3eeb2aaeb75486f1ecba640f67e785288fab17c843858f7a1542c49fa4c295da" exitCode=137 Aug 05 12:43:55.311026 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.310537 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerDied","Data":"b3b192f80fe5ac94ddbe6364b3cfe2c4bafd0963bd75baf74e51504987c60004"} Aug 05 12:43:55.311026 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.310582 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerDied","Data":"3eeb2aaeb75486f1ecba640f67e785288fab17c843858f7a1542c49fa4c295da"} Aug 05 12:43:55.313206 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.313181 2569 generic.go:356] "Generic (PLEG): container finished" podID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerID="9b233a7410a0b6268c358359c45d4cabb4cfe8c6e176a4397b17b8848e115157" exitCode=137 Aug 05 12:43:55.313311 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.313255 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerDied","Data":"9b233a7410a0b6268c358359c45d4cabb4cfe8c6e176a4397b17b8848e115157"} Aug 05 12:43:55.405714 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.405680 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:43:55.412725 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.412703 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:43:55.650843 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.650821 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:43:55.730727 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.730698 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-ui-tls\") pod \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " Aug 05 12:43:55.730887 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.730739 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/6e5f875a-0e37-4775-b4db-4bb00cb808d4-registry-tls\") pod \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " Aug 05 12:43:55.730887 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.730782 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e5f875a-0e37-4775-b4db-4bb00cb808d4-feast-data\") pod \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " Aug 05 12:43:55.730887 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.730810 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls\") pod \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " Aug 05 12:43:55.730887 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.730853 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls\") pod \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " Aug 05 12:43:55.731137 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.731015 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2tmg\" (UniqueName: \"kubernetes.io/projected/6e5f875a-0e37-4775-b4db-4bb00cb808d4-kube-api-access-f2tmg\") pod \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\" (UID: \"6e5f875a-0e37-4775-b4db-4bb00cb808d4\") " Aug 05 12:43:55.731246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.731214 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e5f875a-0e37-4775-b4db-4bb00cb808d4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6e5f875a-0e37-4775-b4db-4bb00cb808d4" (UID: "6e5f875a-0e37-4775-b4db-4bb00cb808d4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:43:55.731371 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.731348 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e5f875a-0e37-4775-b4db-4bb00cb808d4-feast-data" (OuterVolumeSpecName: "feast-data") pod "6e5f875a-0e37-4775-b4db-4bb00cb808d4" (UID: "6e5f875a-0e37-4775-b4db-4bb00cb808d4"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:43:55.731439 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.731381 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/6e5f875a-0e37-4775-b4db-4bb00cb808d4-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:55.733203 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.733175 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "6e5f875a-0e37-4775-b4db-4bb00cb808d4" (UID: "6e5f875a-0e37-4775-b4db-4bb00cb808d4"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:43:55.733316 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.733255 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "6e5f875a-0e37-4775-b4db-4bb00cb808d4" (UID: "6e5f875a-0e37-4775-b4db-4bb00cb808d4"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:43:55.733381 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.733337 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls" (OuterVolumeSpecName: "online-tls") pod "6e5f875a-0e37-4775-b4db-4bb00cb808d4" (UID: "6e5f875a-0e37-4775-b4db-4bb00cb808d4"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:43:55.733455 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.733427 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e5f875a-0e37-4775-b4db-4bb00cb808d4-kube-api-access-f2tmg" (OuterVolumeSpecName: "kube-api-access-f2tmg") pod "6e5f875a-0e37-4775-b4db-4bb00cb808d4" (UID: "6e5f875a-0e37-4775-b4db-4bb00cb808d4"). InnerVolumeSpecName "kube-api-access-f2tmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:43:55.807930 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.807902 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:43:55.832685 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.832627 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:55.832685 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.832657 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6e5f875a-0e37-4775-b4db-4bb00cb808d4-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:55.832685 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.832672 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:55.832685 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.832683 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6e5f875a-0e37-4775-b4db-4bb00cb808d4-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:55.832894 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.832697 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2tmg\" (UniqueName: \"kubernetes.io/projected/6e5f875a-0e37-4775-b4db-4bb00cb808d4-kube-api-access-f2tmg\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:55.933063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.933038 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad7bda7-2959-4a5f-935c-8ee71d561674-feast-data\") pod \"6ad7bda7-2959-4a5f-935c-8ee71d561674\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " Aug 05 12:43:55.933184 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.933074 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhk4w\" (UniqueName: \"kubernetes.io/projected/6ad7bda7-2959-4a5f-935c-8ee71d561674-kube-api-access-vhk4w\") pod \"6ad7bda7-2959-4a5f-935c-8ee71d561674\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " Aug 05 12:43:55.933184 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.933129 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-offline-tls\") pod \"6ad7bda7-2959-4a5f-935c-8ee71d561674\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " Aug 05 12:43:55.933184 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.933155 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-online-tls\") pod \"6ad7bda7-2959-4a5f-935c-8ee71d561674\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " Aug 05 12:43:55.933319 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.933209 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-registry-tls\") pod \"6ad7bda7-2959-4a5f-935c-8ee71d561674\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " Aug 05 12:43:55.933319 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.933279 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls\") pod \"6ad7bda7-2959-4a5f-935c-8ee71d561674\" (UID: \"6ad7bda7-2959-4a5f-935c-8ee71d561674\") " Aug 05 12:43:55.933622 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.933592 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad7bda7-2959-4a5f-935c-8ee71d561674-feast-data" (OuterVolumeSpecName: "feast-data") pod "6ad7bda7-2959-4a5f-935c-8ee71d561674" (UID: "6ad7bda7-2959-4a5f-935c-8ee71d561674"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:43:55.935301 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.935261 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-online-tls" (OuterVolumeSpecName: "online-tls") pod "6ad7bda7-2959-4a5f-935c-8ee71d561674" (UID: "6ad7bda7-2959-4a5f-935c-8ee71d561674"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:43:55.935392 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.935367 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "6ad7bda7-2959-4a5f-935c-8ee71d561674" (UID: "6ad7bda7-2959-4a5f-935c-8ee71d561674"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:43:55.935434 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.935386 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6ad7bda7-2959-4a5f-935c-8ee71d561674" (UID: "6ad7bda7-2959-4a5f-935c-8ee71d561674"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:43:55.935677 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.935649 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "6ad7bda7-2959-4a5f-935c-8ee71d561674" (UID: "6ad7bda7-2959-4a5f-935c-8ee71d561674"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:43:55.935677 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:55.935662 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad7bda7-2959-4a5f-935c-8ee71d561674-kube-api-access-vhk4w" (OuterVolumeSpecName: "kube-api-access-vhk4w") pod "6ad7bda7-2959-4a5f-935c-8ee71d561674" (UID: "6ad7bda7-2959-4a5f-935c-8ee71d561674"). InnerVolumeSpecName "kube-api-access-vhk4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:43:56.034183 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.034159 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad7bda7-2959-4a5f-935c-8ee71d561674-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:56.034183 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.034181 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhk4w\" (UniqueName: \"kubernetes.io/projected/6ad7bda7-2959-4a5f-935c-8ee71d561674-kube-api-access-vhk4w\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:56.034299 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.034191 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:56.034299 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.034199 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:56.034299 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.034206 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:56.034299 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.034215 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ad7bda7-2959-4a5f-935c-8ee71d561674-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:43:56.319548 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.319516 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" Aug 05 12:43:56.319548 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.319525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w" event={"ID":"6ad7bda7-2959-4a5f-935c-8ee71d561674","Type":"ContainerDied","Data":"eba9949dce31b66c89c10393ce0b0b1ab63bfb87e8eab8b1d49fc47c25319eac"} Aug 05 12:43:56.320170 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.319582 2569 scope.go:117] "RemoveContainer" containerID="258fda06fac5eae29463241f5dcebc253faf959b6624c9b6879e7a6f50f2778d" Aug 05 12:43:56.323761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.323628 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" event={"ID":"6e5f875a-0e37-4775-b4db-4bb00cb808d4","Type":"ContainerDied","Data":"b8cf5992c5eb45ec3a04132ed3bcfc640fa96be4c2d2a78ba7ece0ce85ca4d9d"} Aug 05 12:43:56.323761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.323737 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v" Aug 05 12:43:56.331758 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.331739 2569 scope.go:117] "RemoveContainer" containerID="b3b192f80fe5ac94ddbe6364b3cfe2c4bafd0963bd75baf74e51504987c60004" Aug 05 12:43:56.339636 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.339617 2569 scope.go:117] "RemoveContainer" containerID="abbb872534d694d09bf7e4c7ecacf776490c02f2b070070366a5aadf497b64ad" Aug 05 12:43:56.344131 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.344107 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w"] Aug 05 12:43:56.347870 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.347848 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mkg5w"] Aug 05 12:43:56.348306 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.348285 2569 scope.go:117] "RemoveContainer" containerID="3eeb2aaeb75486f1ecba640f67e785288fab17c843858f7a1542c49fa4c295da" Aug 05 12:43:56.355813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.355772 2569 scope.go:117] "RemoveContainer" containerID="980ccba099346ffcf40f45f8ecfc4ff20d4b79731bcd8ca35e0cfe7c69d733a3" Aug 05 12:43:56.357710 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.357692 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v"] Aug 05 12:43:56.361015 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.360993 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-j9f9v"] Aug 05 12:43:56.363581 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.363558 2569 scope.go:117] "RemoveContainer" containerID="1abf3008a3f1b1a11353bfaf8430d93a4a8c045732d13f9d48d41db69388af02" Aug 05 12:43:56.370322 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.370305 2569 scope.go:117] "RemoveContainer" containerID="965d4233a13f0c45df3de9a4e7f7eb02007fed50fac7d9ac0a2fb483867e7eea" Aug 05 12:43:56.377380 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.377366 2569 scope.go:117] "RemoveContainer" containerID="9b233a7410a0b6268c358359c45d4cabb4cfe8c6e176a4397b17b8848e115157" Aug 05 12:43:56.384218 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.384204 2569 scope.go:117] "RemoveContainer" containerID="c45d36dcf33813d4118364f0ae20c4dedd733bacddc21997b6d47818be79b522" Aug 05 12:43:56.391177 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.391157 2569 scope.go:117] "RemoveContainer" containerID="454de08b6ead9410315eec1f19943e6bd7a0e815a48d64471d1e8440ef5e3a59" Aug 05 12:43:56.398092 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:56.398075 2569 scope.go:117] "RemoveContainer" containerID="08b796209618aab424cd3849d436bdd6f08265d88f465b00f9ee51c661f60b62" Aug 05 12:43:57.441514 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:57.441470 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" path="/var/lib/kubelet/pods/6ad7bda7-2959-4a5f-935c-8ee71d561674/volumes" Aug 05 12:43:57.442310 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:43:57.442287 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" path="/var/lib/kubelet/pods/6e5f875a-0e37-4775-b4db-4bb00cb808d4/volumes" Aug 05 12:44:08.047023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:08.046985 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:08.181097 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:08.181064 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:13.849171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:13.849133 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:13.917117 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:13.917090 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:32.865409 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:32.865377 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:32.967118 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:32.967086 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:36.962237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:36.962208 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:37.061922 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.061894 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:37.066813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.066786 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp"] Aug 05 12:44:37.067116 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067103 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="feast-init" Aug 05 12:44:37.067171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067118 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="feast-init" Aug 05 12:44:37.067171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067126 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="ui" Aug 05 12:44:37.067171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067132 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="ui" Aug 05 12:44:37.067171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067141 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="registry" Aug 05 12:44:37.067171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067147 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="registry" Aug 05 12:44:37.067171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067153 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="online" Aug 05 12:44:37.067171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067158 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="online" Aug 05 12:44:37.067171 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067170 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="feast-apply" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067176 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="feast-apply" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067186 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="offline" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067191 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="offline" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067199 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="feast-apply" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067204 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="feast-apply" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067212 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="offline" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067217 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="offline" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067224 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="feast-init" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067229 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="feast-init" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067235 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="ui" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067240 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="ui" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067247 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="online" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067252 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="online" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067299 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="offline" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067308 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="online" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067314 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="online" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067319 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="offline" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067324 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="registry" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067332 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e5f875a-0e37-4775-b4db-4bb00cb808d4" containerName="ui" Aug 05 12:44:37.067411 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.067338 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad7bda7-2959-4a5f-935c-8ee71d561674" containerName="ui" Aug 05 12:44:37.070742 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.070726 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.072823 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.072800 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-cvhq9\"" Aug 05 12:44:37.072823 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.072811 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Aug 05 12:44:37.073145 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.072811 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Aug 05 12:44:37.073145 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.072811 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Aug 05 12:44:37.073145 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.073144 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Aug 05 12:44:37.073313 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.073155 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Aug 05 12:44:37.073313 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.073148 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Aug 05 12:44:37.081026 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.081007 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp"] Aug 05 12:44:37.130219 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.130194 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.130329 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.130229 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj92l\" (UniqueName: \"kubernetes.io/projected/2c133670-8f24-4a39-a13d-1f06584fbc96-kube-api-access-sj92l\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.130329 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.130291 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.130329 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.130324 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.130438 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.130348 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2c133670-8f24-4a39-a13d-1f06584fbc96-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.130438 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.130372 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.186254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.186229 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:37.231408 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.231350 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.231408 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.231387 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj92l\" (UniqueName: \"kubernetes.io/projected/2c133670-8f24-4a39-a13d-1f06584fbc96-kube-api-access-sj92l\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.231408 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.231406 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.231578 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.231429 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.231578 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.231454 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2c133670-8f24-4a39-a13d-1f06584fbc96-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.231578 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.231488 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.231578 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:44:37.231514 2569 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Aug 05 12:44:37.231738 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:44:37.231596 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls podName:2c133670-8f24-4a39-a13d-1f06584fbc96 nodeName:}" failed. No retries permitted until 2026-08-05 12:44:37.731575174 +0000 UTC m=+1542.797167069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls") pod "feast-simple-feast-setup-777d6cc998-79hmp" (UID: "2c133670-8f24-4a39-a13d-1f06584fbc96") : secret "feast-simple-feast-setup-offline-tls" not found Aug 05 12:44:37.231738 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:44:37.231604 2569 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Aug 05 12:44:37.231738 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:44:37.231652 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls podName:2c133670-8f24-4a39-a13d-1f06584fbc96 nodeName:}" failed. No retries permitted until 2026-08-05 12:44:37.7316339 +0000 UTC m=+1542.797225776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls") pod "feast-simple-feast-setup-777d6cc998-79hmp" (UID: "2c133670-8f24-4a39-a13d-1f06584fbc96") : secret "feast-simple-feast-setup-ui-tls" not found Aug 05 12:44:37.231882 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.231866 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2c133670-8f24-4a39-a13d-1f06584fbc96-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.233867 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.233845 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.233973 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.233928 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.238871 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.238851 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj92l\" (UniqueName: \"kubernetes.io/projected/2c133670-8f24-4a39-a13d-1f06584fbc96-kube-api-access-sj92l\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.286207 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.286181 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:37.387225 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.387204 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:37.737387 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.737354 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.737583 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.737400 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.739550 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.739523 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.739704 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.739683 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-79hmp\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:37.990623 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:37.990547 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:38.119606 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:38.119578 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp"] Aug 05 12:44:38.120793 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:44:38.120765 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c133670_8f24_4a39_a13d_1f06584fbc96.slice/crio-01d6e23518f10f05ffb7e8cb724f79da530fdb749a1572ff12a3c9df3d88015d WatchSource:0}: Error finding container 01d6e23518f10f05ffb7e8cb724f79da530fdb749a1572ff12a3c9df3d88015d: Status 404 returned error can't find the container with id 01d6e23518f10f05ffb7e8cb724f79da530fdb749a1572ff12a3c9df3d88015d Aug 05 12:44:38.498474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:38.498437 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerStarted","Data":"689b9f8bd25127d2ebc0121291ce575c6472210572bc6d3daf9b2ba13aa4f6aa"} Aug 05 12:44:38.498689 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:38.498486 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerStarted","Data":"01d6e23518f10f05ffb7e8cb724f79da530fdb749a1572ff12a3c9df3d88015d"} Aug 05 12:44:43.520558 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:43.520523 2569 generic.go:356] "Generic (PLEG): container finished" podID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerID="689b9f8bd25127d2ebc0121291ce575c6472210572bc6d3daf9b2ba13aa4f6aa" exitCode=0 Aug 05 12:44:43.520984 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:43.520574 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerDied","Data":"689b9f8bd25127d2ebc0121291ce575c6472210572bc6d3daf9b2ba13aa4f6aa"} Aug 05 12:44:44.526988 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:44.526939 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerStarted","Data":"ae14295a58c552f55252a912cc9fb69a62848ffd956902352d576abc37276bff"} Aug 05 12:44:49.548387 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:49.548354 2569 generic.go:356] "Generic (PLEG): container finished" podID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerID="ae14295a58c552f55252a912cc9fb69a62848ffd956902352d576abc37276bff" exitCode=0 Aug 05 12:44:49.548774 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:49.548417 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerDied","Data":"ae14295a58c552f55252a912cc9fb69a62848ffd956902352d576abc37276bff"} Aug 05 12:44:50.575128 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:50.575085 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerStarted","Data":"94353ccdecb1966028a6b0e41b238ecda45b916dec4c34176cbdae1ca01db5df"} Aug 05 12:44:50.575615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:50.575151 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerStarted","Data":"843203ce438bef266441c42977bb45f0bb33a675b96366f511f2c799d66a8e99"} Aug 05 12:44:50.575615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:50.575174 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerStarted","Data":"81df2d29ff3d64b5688e1f719ba634f8c4152aa553435f073001d871e9efa6ec"} Aug 05 12:44:50.575615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:50.575192 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerStarted","Data":"aaab2dc4e46cc3e66fb2ab3a404253e0963bde9373d208719c7f3a240627f3d2"} Aug 05 12:44:50.600907 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:50.600828 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podStartSLOduration=13.600805226 podStartE2EDuration="13.600805226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:44:50.599892588 +0000 UTC m=+1555.665484486" watchObservedRunningTime="2026-08-05 12:44:50.600805226 +0000 UTC m=+1555.666397150" Aug 05 12:44:52.991932 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:52.990994 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:52.991932 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:52.991060 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:52.991932 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:52.991082 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:52.992761 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:52.992742 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:52.993208 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:52.993098 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Aug 05 12:44:52.993461 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:52.993391 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.43:6571: connect: connection refused" Aug 05 12:44:52.993461 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:52.993404 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Aug 05 12:44:52.993675 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:52.993654 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.43:8816: connect: connection refused" Aug 05 12:44:54.643197 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:54.643159 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:44:55.992288 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.992213 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.992615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.992332 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.992615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.992572 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.993033 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.992980 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.993033 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.993033 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.993185 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.993055 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.993185 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.993109 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.993247 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.993198 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.993419 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.993409 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:55.997402 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:55.997379 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:56.609630 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:56.609601 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:44:56.612909 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:44:56.612883 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:45:58.615176 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.615140 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp"] Aug 05 12:45:58.618369 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.615518 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="registry" containerID="cri-o://aaab2dc4e46cc3e66fb2ab3a404253e0963bde9373d208719c7f3a240627f3d2" gracePeriod=30 Aug 05 12:45:58.618369 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.615574 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="offline" containerID="cri-o://843203ce438bef266441c42977bb45f0bb33a675b96366f511f2c799d66a8e99" gracePeriod=30 Aug 05 12:45:58.618369 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.615675 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="online" containerID="cri-o://81df2d29ff3d64b5688e1f719ba634f8c4152aa553435f073001d871e9efa6ec" gracePeriod=30 Aug 05 12:45:58.618369 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.615724 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="ui" containerID="cri-o://94353ccdecb1966028a6b0e41b238ecda45b916dec4c34176cbdae1ca01db5df" gracePeriod=30 Aug 05 12:45:58.618369 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.617103 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:45:58.718307 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.718267 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:45:58.818759 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.818729 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:45:58.905168 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.905050 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:45:58.907804 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.907775 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x"] Aug 05 12:45:58.911739 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.911722 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:58.913708 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.913687 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-c87rv\"" Aug 05 12:45:58.922327 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.922304 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x"] Aug 05 12:45:58.999146 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.999113 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:58.999146 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.999156 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:58.999448 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.999180 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xwpw\" (UniqueName: \"kubernetes.io/projected/028ef312-8b95-4253-a9f7-3b92695a6e79-kube-api-access-6xwpw\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:58.999448 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.999297 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:58.999448 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.999333 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:58.999448 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:58.999377 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028ef312-8b95-4253-a9f7-3b92695a6e79-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.000369 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.000345 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:45:59.099841 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.099812 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.099978 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.099850 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.099978 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.099870 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xwpw\" (UniqueName: \"kubernetes.io/projected/028ef312-8b95-4253-a9f7-3b92695a6e79-kube-api-access-6xwpw\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.099978 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.099911 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.099978 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.099934 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.099978 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.099975 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028ef312-8b95-4253-a9f7-3b92695a6e79-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.100472 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.100438 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028ef312-8b95-4253-a9f7-3b92695a6e79-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.102437 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.102401 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:45:59.102795 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.102775 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.102997 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.102975 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.103102 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.103081 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.103215 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.103199 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.108254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.108229 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xwpw\" (UniqueName: \"kubernetes.io/projected/028ef312-8b95-4253-a9f7-3b92695a6e79-kube-api-access-6xwpw\") pod \"feast-simple-feast-setup-777d6cc998-9hd7x\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.201384 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.201311 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:45:59.221486 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.221456 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:45:59.354733 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.354710 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x"] Aug 05 12:45:59.356375 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:45:59.356353 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod028ef312_8b95_4253_a9f7_3b92695a6e79.slice/crio-ff91c641115bc4a0e8eacb6b57b8c7000fb2e6cf7777b68054d4edf1958933d8 WatchSource:0}: Error finding container ff91c641115bc4a0e8eacb6b57b8c7000fb2e6cf7777b68054d4edf1958933d8: Status 404 returned error can't find the container with id ff91c641115bc4a0e8eacb6b57b8c7000fb2e6cf7777b68054d4edf1958933d8 Aug 05 12:45:59.873107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.873075 2569 generic.go:356] "Generic (PLEG): container finished" podID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerID="94353ccdecb1966028a6b0e41b238ecda45b916dec4c34176cbdae1ca01db5df" exitCode=0 Aug 05 12:45:59.873107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.873101 2569 generic.go:356] "Generic (PLEG): container finished" podID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerID="81df2d29ff3d64b5688e1f719ba634f8c4152aa553435f073001d871e9efa6ec" exitCode=0 Aug 05 12:45:59.873571 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.873146 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerDied","Data":"94353ccdecb1966028a6b0e41b238ecda45b916dec4c34176cbdae1ca01db5df"} Aug 05 12:45:59.873571 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.873195 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerDied","Data":"81df2d29ff3d64b5688e1f719ba634f8c4152aa553435f073001d871e9efa6ec"} Aug 05 12:45:59.874632 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.874608 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerStarted","Data":"224fc67aed64cd3c56acd5e695b140d4f82363d03a6d94d1aafe7843530a3e2a"} Aug 05 12:45:59.874772 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:45:59.874640 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerStarted","Data":"ff91c641115bc4a0e8eacb6b57b8c7000fb2e6cf7777b68054d4edf1958933d8"} Aug 05 12:46:04.896927 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:04.896895 2569 generic.go:356] "Generic (PLEG): container finished" podID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerID="224fc67aed64cd3c56acd5e695b140d4f82363d03a6d94d1aafe7843530a3e2a" exitCode=0 Aug 05 12:46:04.897413 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:04.896982 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerDied","Data":"224fc67aed64cd3c56acd5e695b140d4f82363d03a6d94d1aafe7843530a3e2a"} Aug 05 12:46:05.903068 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:05.903034 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerStarted","Data":"2cbedeb224e81cef6cdf0141e422b3261a8a6137dd1d556b81aeb9dc0465c99b"} Aug 05 12:46:05.993385 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:05.993336 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Aug 05 12:46:06.610556 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:06.610513 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Aug 05 12:46:10.930308 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:10.930276 2569 generic.go:356] "Generic (PLEG): container finished" podID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerID="2cbedeb224e81cef6cdf0141e422b3261a8a6137dd1d556b81aeb9dc0465c99b" exitCode=0 Aug 05 12:46:10.930664 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:10.930326 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerDied","Data":"2cbedeb224e81cef6cdf0141e422b3261a8a6137dd1d556b81aeb9dc0465c99b"} Aug 05 12:46:11.946514 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:11.946472 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerStarted","Data":"273893ccf394e58e8d4c1b239eb0bfb68e6d76097cc0398190ceba59e47bcbb5"} Aug 05 12:46:11.947023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:11.946535 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerStarted","Data":"b878c964dcf613b2c91207477562722229ecbce641b1dd0f039a56ec39838051"} Aug 05 12:46:11.947023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:11.946564 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerStarted","Data":"e034b536e1d7c5815bd9f08b604591c0242fbb4c1256321acfbbff2d705d9be3"} Aug 05 12:46:11.947023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:11.946581 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerStarted","Data":"89767331317a6f5f88a087b086ee2a51d1682411b254dead5c8b591a5c3ac3f6"} Aug 05 12:46:11.970144 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:11.970082 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podStartSLOduration=13.970037094 podStartE2EDuration="13.970037094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:46:11.968355996 +0000 UTC m=+1637.033947920" watchObservedRunningTime="2026-08-05 12:46:11.970037094 +0000 UTC m=+1637.035628993" Aug 05 12:46:14.222281 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:14.222242 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:14.222854 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:14.222836 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:14.223029 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:14.223006 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:14.223157 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:14.223037 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:14.225059 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:14.225028 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.44:8816: connect: connection refused" Aug 05 12:46:14.225269 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:14.225071 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Aug 05 12:46:14.225448 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:14.225078 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Aug 05 12:46:14.225562 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:14.225114 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.44:6571: connect: connection refused" Aug 05 12:46:15.993316 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:15.993246 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Aug 05 12:46:16.610704 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:16.610650 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Aug 05 12:46:17.222753 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.222723 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.223205 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.222833 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.223205 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.223145 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.223560 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.223532 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.223746 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.223728 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.223876 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.223751 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.223876 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.223805 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.224014 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.224001 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.224056 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.224042 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.227812 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.227795 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.981694 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.981663 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:17.984689 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:17.984670 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:46:25.993792 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:25.993745 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.43:8443: connect: connection refused" Aug 05 12:46:25.994427 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:25.993880 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:46:26.609833 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:26.609791 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="online" probeResult="failure" output="Get \"https://10.132.0.43:6567/health\": dial tcp 10.132.0.43:6567: connect: connection refused" Aug 05 12:46:26.610034 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:26.609905 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:46:29.028974 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.028928 2569 generic.go:356] "Generic (PLEG): container finished" podID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerID="843203ce438bef266441c42977bb45f0bb33a675b96366f511f2c799d66a8e99" exitCode=137 Aug 05 12:46:29.028974 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.028966 2569 generic.go:356] "Generic (PLEG): container finished" podID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerID="aaab2dc4e46cc3e66fb2ab3a404253e0963bde9373d208719c7f3a240627f3d2" exitCode=137 Aug 05 12:46:29.029390 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.028982 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerDied","Data":"843203ce438bef266441c42977bb45f0bb33a675b96366f511f2c799d66a8e99"} Aug 05 12:46:29.029390 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.029037 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerDied","Data":"aaab2dc4e46cc3e66fb2ab3a404253e0963bde9373d208719c7f3a240627f3d2"} Aug 05 12:46:29.262220 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.262198 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:46:29.357634 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.357569 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls\") pod \"2c133670-8f24-4a39-a13d-1f06584fbc96\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " Aug 05 12:46:29.357634 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.357610 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-online-tls\") pod \"2c133670-8f24-4a39-a13d-1f06584fbc96\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " Aug 05 12:46:29.357853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.357645 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj92l\" (UniqueName: \"kubernetes.io/projected/2c133670-8f24-4a39-a13d-1f06584fbc96-kube-api-access-sj92l\") pod \"2c133670-8f24-4a39-a13d-1f06584fbc96\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " Aug 05 12:46:29.357853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.357668 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2c133670-8f24-4a39-a13d-1f06584fbc96-feast-data\") pod \"2c133670-8f24-4a39-a13d-1f06584fbc96\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " Aug 05 12:46:29.357853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.357740 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-registry-tls\") pod \"2c133670-8f24-4a39-a13d-1f06584fbc96\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " Aug 05 12:46:29.357853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.357789 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls\") pod \"2c133670-8f24-4a39-a13d-1f06584fbc96\" (UID: \"2c133670-8f24-4a39-a13d-1f06584fbc96\") " Aug 05 12:46:29.358268 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.358242 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c133670-8f24-4a39-a13d-1f06584fbc96-feast-data" (OuterVolumeSpecName: "feast-data") pod "2c133670-8f24-4a39-a13d-1f06584fbc96" (UID: "2c133670-8f24-4a39-a13d-1f06584fbc96"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:46:29.359897 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.359869 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-online-tls" (OuterVolumeSpecName: "online-tls") pod "2c133670-8f24-4a39-a13d-1f06584fbc96" (UID: "2c133670-8f24-4a39-a13d-1f06584fbc96"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:46:29.360252 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.360228 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c133670-8f24-4a39-a13d-1f06584fbc96-kube-api-access-sj92l" (OuterVolumeSpecName: "kube-api-access-sj92l") pod "2c133670-8f24-4a39-a13d-1f06584fbc96" (UID: "2c133670-8f24-4a39-a13d-1f06584fbc96"). InnerVolumeSpecName "kube-api-access-sj92l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:46:29.360335 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.360290 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2c133670-8f24-4a39-a13d-1f06584fbc96" (UID: "2c133670-8f24-4a39-a13d-1f06584fbc96"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:46:29.360335 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.360301 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2c133670-8f24-4a39-a13d-1f06584fbc96" (UID: "2c133670-8f24-4a39-a13d-1f06584fbc96"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:46:29.360335 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.360320 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2c133670-8f24-4a39-a13d-1f06584fbc96" (UID: "2c133670-8f24-4a39-a13d-1f06584fbc96"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:46:29.458892 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.458872 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:46:29.458991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.458893 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj92l\" (UniqueName: \"kubernetes.io/projected/2c133670-8f24-4a39-a13d-1f06584fbc96-kube-api-access-sj92l\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:46:29.458991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.458903 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2c133670-8f24-4a39-a13d-1f06584fbc96-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:46:29.458991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.458912 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:46:29.458991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.458919 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:46:29.458991 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:29.458926 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2c133670-8f24-4a39-a13d-1f06584fbc96-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:46:30.035612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.035583 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" Aug 05 12:46:30.035612 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.035591 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp" event={"ID":"2c133670-8f24-4a39-a13d-1f06584fbc96","Type":"ContainerDied","Data":"01d6e23518f10f05ffb7e8cb724f79da530fdb749a1572ff12a3c9df3d88015d"} Aug 05 12:46:30.036144 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.035647 2569 scope.go:117] "RemoveContainer" containerID="94353ccdecb1966028a6b0e41b238ecda45b916dec4c34176cbdae1ca01db5df" Aug 05 12:46:30.044506 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.044487 2569 scope.go:117] "RemoveContainer" containerID="843203ce438bef266441c42977bb45f0bb33a675b96366f511f2c799d66a8e99" Aug 05 12:46:30.052445 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.052412 2569 scope.go:117] "RemoveContainer" containerID="81df2d29ff3d64b5688e1f719ba634f8c4152aa553435f073001d871e9efa6ec" Aug 05 12:46:30.052546 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.052467 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp"] Aug 05 12:46:30.056169 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.056149 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-79hmp"] Aug 05 12:46:30.060114 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.060100 2569 scope.go:117] "RemoveContainer" containerID="aaab2dc4e46cc3e66fb2ab3a404253e0963bde9373d208719c7f3a240627f3d2" Aug 05 12:46:30.067307 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.067288 2569 scope.go:117] "RemoveContainer" containerID="ae14295a58c552f55252a912cc9fb69a62848ffd956902352d576abc37276bff" Aug 05 12:46:30.073926 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:30.073911 2569 scope.go:117] "RemoveContainer" containerID="689b9f8bd25127d2ebc0121291ce575c6472210572bc6d3daf9b2ba13aa4f6aa" Aug 05 12:46:31.441229 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:31.441199 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" path="/var/lib/kubelet/pods/2c133670-8f24-4a39-a13d-1f06584fbc96/volumes" Aug 05 12:46:45.413246 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:46:45.413210 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:47:20.037759 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.037676 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:47:20.395946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.395856 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m"] Aug 05 12:47:20.396356 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396337 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="feast-apply" Aug 05 12:47:20.396458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396357 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="feast-apply" Aug 05 12:47:20.396458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396386 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="registry" Aug 05 12:47:20.396458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396394 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="registry" Aug 05 12:47:20.396458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396426 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="feast-init" Aug 05 12:47:20.396458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396436 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="feast-init" Aug 05 12:47:20.396458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396445 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="ui" Aug 05 12:47:20.396458 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396453 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="ui" Aug 05 12:47:20.396813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396467 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="online" Aug 05 12:47:20.396813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396475 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="online" Aug 05 12:47:20.396813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396508 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="offline" Aug 05 12:47:20.396813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396517 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="offline" Aug 05 12:47:20.396813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396604 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="ui" Aug 05 12:47:20.396813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396615 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="online" Aug 05 12:47:20.396813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396626 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="registry" Aug 05 12:47:20.396813 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.396637 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c133670-8f24-4a39-a13d-1f06584fbc96" containerName="offline" Aug 05 12:47:20.400363 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.400343 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.403275 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.403254 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Aug 05 12:47:20.403388 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.403254 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Aug 05 12:47:20.403388 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.403300 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Aug 05 12:47:20.403388 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.403253 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Aug 05 12:47:20.403541 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.403465 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Aug 05 12:47:20.403731 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.403715 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Aug 05 12:47:20.403782 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.403765 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-rmcr2\"" Aug 05 12:47:20.416650 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.416631 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m"] Aug 05 12:47:20.457834 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.457798 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gxmr\" (UniqueName: \"kubernetes.io/projected/31cc4365-94ca-403a-84d9-952af97a420a-kube-api-access-7gxmr\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.457937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.457854 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31cc4365-94ca-403a-84d9-952af97a420a-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.457937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.457910 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.458062 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.457937 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.458062 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.457982 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.458062 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.458030 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/31cc4365-94ca-403a-84d9-952af97a420a-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.558823 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.558798 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.558944 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.558834 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/31cc4365-94ca-403a-84d9-952af97a420a-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.558944 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.558853 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gxmr\" (UniqueName: \"kubernetes.io/projected/31cc4365-94ca-403a-84d9-952af97a420a-kube-api-access-7gxmr\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.558944 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:47:20.558931 2569 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Aug 05 12:47:20.559152 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.558996 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31cc4365-94ca-403a-84d9-952af97a420a-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.559152 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:47:20.559013 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls podName:31cc4365-94ca-403a-84d9-952af97a420a nodeName:}" failed. No retries permitted until 2026-08-05 12:47:21.058998458 +0000 UTC m=+1706.124590334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls") pod "feast-simple-feast-remote-setup-db6955c6c-8zk8m" (UID: "31cc4365-94ca-403a-84d9-952af97a420a") : secret "feast-simple-feast-remote-setup-online-tls" not found Aug 05 12:47:20.559152 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.559106 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.559152 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.559143 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.559377 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:47:20.559191 2569 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Aug 05 12:47:20.559377 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:47:20.559255 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls podName:31cc4365-94ca-403a-84d9-952af97a420a nodeName:}" failed. No retries permitted until 2026-08-05 12:47:21.059239002 +0000 UTC m=+1706.124830891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls") pod "feast-simple-feast-remote-setup-db6955c6c-8zk8m" (UID: "31cc4365-94ca-403a-84d9-952af97a420a") : secret "feast-simple-feast-remote-setup-offline-tls" not found Aug 05 12:47:20.559377 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.559303 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31cc4365-94ca-403a-84d9-952af97a420a-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.559623 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.559604 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/31cc4365-94ca-403a-84d9-952af97a420a-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.561474 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.561455 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:20.567419 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:20.567391 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gxmr\" (UniqueName: \"kubernetes.io/projected/31cc4365-94ca-403a-84d9-952af97a420a-kube-api-access-7gxmr\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:21.062227 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:21.062191 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:21.062604 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:21.062243 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:21.064549 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:21.064521 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:21.064667 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:21.064525 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-8zk8m\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:21.309393 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:21.309354 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:21.442579 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:47:21.442548 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31cc4365_94ca_403a_84d9_952af97a420a.slice/crio-23488cadee1b83b060f4db19a3f490055003e0bfaf4e557e15d908509a8883d2 WatchSource:0}: Error finding container 23488cadee1b83b060f4db19a3f490055003e0bfaf4e557e15d908509a8883d2: Status 404 returned error can't find the container with id 23488cadee1b83b060f4db19a3f490055003e0bfaf4e557e15d908509a8883d2 Aug 05 12:47:21.442874 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:21.442856 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m"] Aug 05 12:47:21.994553 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:21.994521 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:47:22.256107 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:22.256076 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerStarted","Data":"33ae43382aacf2354d435971942fc847dc3c29869163fca787f80bd2fac9cbf0"} Aug 05 12:47:22.256486 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:22.256127 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerStarted","Data":"23488cadee1b83b060f4db19a3f490055003e0bfaf4e557e15d908509a8883d2"} Aug 05 12:47:26.273504 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:26.273474 2569 generic.go:356] "Generic (PLEG): container finished" podID="31cc4365-94ca-403a-84d9-952af97a420a" containerID="33ae43382aacf2354d435971942fc847dc3c29869163fca787f80bd2fac9cbf0" exitCode=0 Aug 05 12:47:26.273820 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:26.273536 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerDied","Data":"33ae43382aacf2354d435971942fc847dc3c29869163fca787f80bd2fac9cbf0"} Aug 05 12:47:27.279051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:27.279015 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerStarted","Data":"417aeac100c090a34bd0cc7f048c797dea6aa5c2373f4476c036762630d2acdc"} Aug 05 12:47:32.303019 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:32.302985 2569 generic.go:356] "Generic (PLEG): container finished" podID="31cc4365-94ca-403a-84d9-952af97a420a" containerID="417aeac100c090a34bd0cc7f048c797dea6aa5c2373f4476c036762630d2acdc" exitCode=0 Aug 05 12:47:32.303374 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:32.303060 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerDied","Data":"417aeac100c090a34bd0cc7f048c797dea6aa5c2373f4476c036762630d2acdc"} Aug 05 12:47:33.311890 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:33.311852 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerStarted","Data":"8588b86fc17776c12d3424eb1fe52823f47b593ea7314435a6575cff0f8d3a53"} Aug 05 12:47:33.312407 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:33.311912 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerStarted","Data":"e047d3c744638a750eea64d43104d5531872a6488f43ca3a9c82578479bbed98"} Aug 05 12:47:33.312407 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:33.311930 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerStarted","Data":"663d8a038a56e6bfac662891057648ac8bd8e5a6369f55315163fae7c2064091"} Aug 05 12:47:33.336088 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:33.336010 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podStartSLOduration=13.335991128 podStartE2EDuration="13.335991128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:47:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:47:33.33458268 +0000 UTC m=+1718.400174577" watchObservedRunningTime="2026-08-05 12:47:33.335991128 +0000 UTC m=+1718.401583027" Aug 05 12:47:36.309750 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:36.309703 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:36.310237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:36.309847 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:36.310237 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:36.309896 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:36.311106 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:36.311073 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.45:8816: connect: connection refused" Aug 05 12:47:36.311241 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:36.311104 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Aug 05 12:47:36.311241 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:36.311123 2569 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Aug 05 12:47:39.310937 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.310908 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:39.311415 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.311039 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:39.311490 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.311431 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:39.311490 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.311471 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:39.311905 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.311886 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:39.311997 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.311971 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:39.317779 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.317758 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:39.341225 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.341202 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:47:39.345070 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:47:39.345050 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:48:41.475885 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.475845 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:48:41.478224 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.478199 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m"] Aug 05 12:48:41.478511 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.478461 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="online" containerID="cri-o://663d8a038a56e6bfac662891057648ac8bd8e5a6369f55315163fae7c2064091" gracePeriod=30 Aug 05 12:48:41.478841 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.478563 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="ui" containerID="cri-o://8588b86fc17776c12d3424eb1fe52823f47b593ea7314435a6575cff0f8d3a53" gracePeriod=30 Aug 05 12:48:41.478946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.478576 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="offline" containerID="cri-o://e047d3c744638a750eea64d43104d5531872a6488f43ca3a9c82578479bbed98" gracePeriod=30 Aug 05 12:48:41.535309 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.535277 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:48:41.621643 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.621614 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:48:41.627541 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.627514 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x"] Aug 05 12:48:41.627758 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.627723 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="registry" containerID="cri-o://89767331317a6f5f88a087b086ee2a51d1682411b254dead5c8b591a5c3ac3f6" gracePeriod=30 Aug 05 12:48:41.627849 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.627785 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="online" containerID="cri-o://e034b536e1d7c5815bd9f08b604591c0242fbb4c1256321acfbbff2d705d9be3" gracePeriod=30 Aug 05 12:48:41.627849 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.627785 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="offline" containerID="cri-o://b878c964dcf613b2c91207477562722229ecbce641b1dd0f039a56ec39838051" gracePeriod=30 Aug 05 12:48:41.628315 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.628102 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="ui" containerID="cri-o://273893ccf394e58e8d4c1b239eb0bfb68e6d76097cc0398190ceba59e47bcbb5" gracePeriod=30 Aug 05 12:48:41.723304 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.723255 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:48:41.779606 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:41.779566 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:48:42.603511 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:42.603427 2569 generic.go:356] "Generic (PLEG): container finished" podID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerID="273893ccf394e58e8d4c1b239eb0bfb68e6d76097cc0398190ceba59e47bcbb5" exitCode=0 Aug 05 12:48:42.603511 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:42.603453 2569 generic.go:356] "Generic (PLEG): container finished" podID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerID="e034b536e1d7c5815bd9f08b604591c0242fbb4c1256321acfbbff2d705d9be3" exitCode=0 Aug 05 12:48:42.603511 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:42.603499 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerDied","Data":"273893ccf394e58e8d4c1b239eb0bfb68e6d76097cc0398190ceba59e47bcbb5"} Aug 05 12:48:42.604016 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:42.603542 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerDied","Data":"e034b536e1d7c5815bd9f08b604591c0242fbb4c1256321acfbbff2d705d9be3"} Aug 05 12:48:42.606014 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:42.605993 2569 generic.go:356] "Generic (PLEG): container finished" podID="31cc4365-94ca-403a-84d9-952af97a420a" containerID="8588b86fc17776c12d3424eb1fe52823f47b593ea7314435a6575cff0f8d3a53" exitCode=0 Aug 05 12:48:42.606014 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:42.606012 2569 generic.go:356] "Generic (PLEG): container finished" podID="31cc4365-94ca-403a-84d9-952af97a420a" containerID="663d8a038a56e6bfac662891057648ac8bd8e5a6369f55315163fae7c2064091" exitCode=0 Aug 05 12:48:42.606166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:42.606067 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerDied","Data":"8588b86fc17776c12d3424eb1fe52823f47b593ea7314435a6575cff0f8d3a53"} Aug 05 12:48:42.606166 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:42.606094 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerDied","Data":"663d8a038a56e6bfac662891057648ac8bd8e5a6369f55315163fae7c2064091"} Aug 05 12:48:46.805471 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:46.805429 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:48:47.224473 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:47.224344 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Aug 05 12:48:47.982720 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:47.982677 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Aug 05 12:48:49.311825 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:49.311784 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Aug 05 12:48:49.341635 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:49.341606 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Aug 05 12:48:50.757816 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:50.757779 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:48:55.433293 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:55.433263 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:48:55.441561 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:55.441531 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:48:57.224211 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:57.224164 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Aug 05 12:48:57.981917 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:57.981876 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Aug 05 12:48:59.312296 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:59.312253 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Aug 05 12:48:59.341916 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:48:59.341875 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Aug 05 12:49:07.224437 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:07.224390 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.44:8443: connect: connection refused" Aug 05 12:49:07.224894 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:07.224535 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:49:07.981860 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:07.981819 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="online" probeResult="failure" output="Get \"https://10.132.0.44:6567/health\": dial tcp 10.132.0.44:6567: connect: connection refused" Aug 05 12:49:07.982078 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:07.981970 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:49:09.311771 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:09.311728 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.45:8443: connect: connection refused" Aug 05 12:49:09.312273 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:09.311863 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:49:09.341337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:09.341302 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.45:6567/health\": dial tcp 10.132.0.45:6567: connect: connection refused" Aug 05 12:49:09.341482 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:09.341399 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:49:11.729713 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:11.729681 2569 generic.go:356] "Generic (PLEG): container finished" podID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerID="b878c964dcf613b2c91207477562722229ecbce641b1dd0f039a56ec39838051" exitCode=137 Aug 05 12:49:11.729713 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:11.729711 2569 generic.go:356] "Generic (PLEG): container finished" podID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerID="89767331317a6f5f88a087b086ee2a51d1682411b254dead5c8b591a5c3ac3f6" exitCode=137 Aug 05 12:49:11.730145 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:11.729750 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerDied","Data":"b878c964dcf613b2c91207477562722229ecbce641b1dd0f039a56ec39838051"} Aug 05 12:49:11.730145 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:11.729792 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerDied","Data":"89767331317a6f5f88a087b086ee2a51d1682411b254dead5c8b591a5c3ac3f6"} Aug 05 12:49:11.732389 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:11.732364 2569 generic.go:356] "Generic (PLEG): container finished" podID="31cc4365-94ca-403a-84d9-952af97a420a" containerID="e047d3c744638a750eea64d43104d5531872a6488f43ca3a9c82578479bbed98" exitCode=137 Aug 05 12:49:11.732505 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:11.732433 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerDied","Data":"e047d3c744638a750eea64d43104d5531872a6488f43ca3a9c82578479bbed98"} Aug 05 12:49:12.130640 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.130613 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:49:12.222539 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.222511 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls\") pod \"31cc4365-94ca-403a-84d9-952af97a420a\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " Aug 05 12:49:12.222539 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.222545 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-ui-tls\") pod \"31cc4365-94ca-403a-84d9-952af97a420a\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " Aug 05 12:49:12.222765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.222579 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gxmr\" (UniqueName: \"kubernetes.io/projected/31cc4365-94ca-403a-84d9-952af97a420a-kube-api-access-7gxmr\") pod \"31cc4365-94ca-403a-84d9-952af97a420a\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " Aug 05 12:49:12.222765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.222597 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31cc4365-94ca-403a-84d9-952af97a420a-feast-data\") pod \"31cc4365-94ca-403a-84d9-952af97a420a\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " Aug 05 12:49:12.222765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.222659 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/31cc4365-94ca-403a-84d9-952af97a420a-registry-tls\") pod \"31cc4365-94ca-403a-84d9-952af97a420a\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " Aug 05 12:49:12.222765 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.222716 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls\") pod \"31cc4365-94ca-403a-84d9-952af97a420a\" (UID: \"31cc4365-94ca-403a-84d9-952af97a420a\") " Aug 05 12:49:12.223373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.223330 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31cc4365-94ca-403a-84d9-952af97a420a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "31cc4365-94ca-403a-84d9-952af97a420a" (UID: "31cc4365-94ca-403a-84d9-952af97a420a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 12:49:12.223538 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.223492 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cc4365-94ca-403a-84d9-952af97a420a-feast-data" (OuterVolumeSpecName: "feast-data") pod "31cc4365-94ca-403a-84d9-952af97a420a" (UID: "31cc4365-94ca-403a-84d9-952af97a420a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:49:12.224963 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.224926 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "31cc4365-94ca-403a-84d9-952af97a420a" (UID: "31cc4365-94ca-403a-84d9-952af97a420a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:49:12.225200 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.225172 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31cc4365-94ca-403a-84d9-952af97a420a-kube-api-access-7gxmr" (OuterVolumeSpecName: "kube-api-access-7gxmr") pod "31cc4365-94ca-403a-84d9-952af97a420a" (UID: "31cc4365-94ca-403a-84d9-952af97a420a"). InnerVolumeSpecName "kube-api-access-7gxmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:49:12.225281 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.225269 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "31cc4365-94ca-403a-84d9-952af97a420a" (UID: "31cc4365-94ca-403a-84d9-952af97a420a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:49:12.225326 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.225290 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls" (OuterVolumeSpecName: "online-tls") pod "31cc4365-94ca-403a-84d9-952af97a420a" (UID: "31cc4365-94ca-403a-84d9-952af97a420a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:49:12.267719 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.267696 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:49:12.324167 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324141 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028ef312-8b95-4253-a9f7-3b92695a6e79-feast-data\") pod \"028ef312-8b95-4253-a9f7-3b92695a6e79\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " Aug 05 12:49:12.324287 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324185 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-registry-tls\") pod \"028ef312-8b95-4253-a9f7-3b92695a6e79\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " Aug 05 12:49:12.324287 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324206 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xwpw\" (UniqueName: \"kubernetes.io/projected/028ef312-8b95-4253-a9f7-3b92695a6e79-kube-api-access-6xwpw\") pod \"028ef312-8b95-4253-a9f7-3b92695a6e79\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " Aug 05 12:49:12.324418 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324376 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-online-tls\") pod \"028ef312-8b95-4253-a9f7-3b92695a6e79\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " Aug 05 12:49:12.324418 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324414 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-ui-tls\") pod \"028ef312-8b95-4253-a9f7-3b92695a6e79\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " Aug 05 12:49:12.324535 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324434 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-offline-tls\") pod \"028ef312-8b95-4253-a9f7-3b92695a6e79\" (UID: \"028ef312-8b95-4253-a9f7-3b92695a6e79\") " Aug 05 12:49:12.324739 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324717 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.324739 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324723 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/028ef312-8b95-4253-a9f7-3b92695a6e79-feast-data" (OuterVolumeSpecName: "feast-data") pod "028ef312-8b95-4253-a9f7-3b92695a6e79" (UID: "028ef312-8b95-4253-a9f7-3b92695a6e79"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:49:12.324739 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324741 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.324996 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324755 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gxmr\" (UniqueName: \"kubernetes.io/projected/31cc4365-94ca-403a-84d9-952af97a420a-kube-api-access-7gxmr\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.324996 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324769 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/31cc4365-94ca-403a-84d9-952af97a420a-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.324996 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324782 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/31cc4365-94ca-403a-84d9-952af97a420a-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.324996 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.324796 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/31cc4365-94ca-403a-84d9-952af97a420a-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.326500 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.326470 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "028ef312-8b95-4253-a9f7-3b92695a6e79" (UID: "028ef312-8b95-4253-a9f7-3b92695a6e79"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:49:12.326592 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.326480 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-online-tls" (OuterVolumeSpecName: "online-tls") pod "028ef312-8b95-4253-a9f7-3b92695a6e79" (UID: "028ef312-8b95-4253-a9f7-3b92695a6e79"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:49:12.326592 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.326564 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "028ef312-8b95-4253-a9f7-3b92695a6e79" (UID: "028ef312-8b95-4253-a9f7-3b92695a6e79"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:49:12.326747 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.326728 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/028ef312-8b95-4253-a9f7-3b92695a6e79-kube-api-access-6xwpw" (OuterVolumeSpecName: "kube-api-access-6xwpw") pod "028ef312-8b95-4253-a9f7-3b92695a6e79" (UID: "028ef312-8b95-4253-a9f7-3b92695a6e79"). InnerVolumeSpecName "kube-api-access-6xwpw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:49:12.326747 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.326737 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "028ef312-8b95-4253-a9f7-3b92695a6e79" (UID: "028ef312-8b95-4253-a9f7-3b92695a6e79"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 12:49:12.426158 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.426132 2569 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-offline-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.426158 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.426156 2569 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/028ef312-8b95-4253-a9f7-3b92695a6e79-feast-data\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.426303 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.426169 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-registry-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.426303 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.426181 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xwpw\" (UniqueName: \"kubernetes.io/projected/028ef312-8b95-4253-a9f7-3b92695a6e79-kube-api-access-6xwpw\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.426303 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.426195 2569 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-online-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.426303 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.426207 2569 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/028ef312-8b95-4253-a9f7-3b92695a6e79-ui-tls\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:12.739470 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.739387 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" event={"ID":"028ef312-8b95-4253-a9f7-3b92695a6e79","Type":"ContainerDied","Data":"ff91c641115bc4a0e8eacb6b57b8c7000fb2e6cf7777b68054d4edf1958933d8"} Aug 05 12:49:12.739470 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.739455 2569 scope.go:117] "RemoveContainer" containerID="273893ccf394e58e8d4c1b239eb0bfb68e6d76097cc0398190ceba59e47bcbb5" Aug 05 12:49:12.740064 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.739396 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x" Aug 05 12:49:12.742205 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.742187 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" Aug 05 12:49:12.742205 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.742173 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m" event={"ID":"31cc4365-94ca-403a-84d9-952af97a420a","Type":"ContainerDied","Data":"23488cadee1b83b060f4db19a3f490055003e0bfaf4e557e15d908509a8883d2"} Aug 05 12:49:12.749324 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.749308 2569 scope.go:117] "RemoveContainer" containerID="b878c964dcf613b2c91207477562722229ecbce641b1dd0f039a56ec39838051" Aug 05 12:49:12.757062 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.757044 2569 scope.go:117] "RemoveContainer" containerID="e034b536e1d7c5815bd9f08b604591c0242fbb4c1256321acfbbff2d705d9be3" Aug 05 12:49:12.764255 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.764150 2569 scope.go:117] "RemoveContainer" containerID="89767331317a6f5f88a087b086ee2a51d1682411b254dead5c8b591a5c3ac3f6" Aug 05 12:49:12.765595 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.765488 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x"] Aug 05 12:49:12.766907 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.766886 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-9hd7x"] Aug 05 12:49:12.772204 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.772186 2569 scope.go:117] "RemoveContainer" containerID="2cbedeb224e81cef6cdf0141e422b3261a8a6137dd1d556b81aeb9dc0465c99b" Aug 05 12:49:12.777552 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.777531 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m"] Aug 05 12:49:12.779337 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.779324 2569 scope.go:117] "RemoveContainer" containerID="224fc67aed64cd3c56acd5e695b140d4f82363d03a6d94d1aafe7843530a3e2a" Aug 05 12:49:12.782946 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.782925 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-8zk8m"] Aug 05 12:49:12.786447 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.786432 2569 scope.go:117] "RemoveContainer" containerID="8588b86fc17776c12d3424eb1fe52823f47b593ea7314435a6575cff0f8d3a53" Aug 05 12:49:12.793477 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.793461 2569 scope.go:117] "RemoveContainer" containerID="e047d3c744638a750eea64d43104d5531872a6488f43ca3a9c82578479bbed98" Aug 05 12:49:12.800486 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.800470 2569 scope.go:117] "RemoveContainer" containerID="663d8a038a56e6bfac662891057648ac8bd8e5a6369f55315163fae7c2064091" Aug 05 12:49:12.807735 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.807717 2569 scope.go:117] "RemoveContainer" containerID="417aeac100c090a34bd0cc7f048c797dea6aa5c2373f4476c036762630d2acdc" Aug 05 12:49:12.814340 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:12.814322 2569 scope.go:117] "RemoveContainer" containerID="33ae43382aacf2354d435971942fc847dc3c29869163fca787f80bd2fac9cbf0" Aug 05 12:49:13.441435 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:13.441405 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" path="/var/lib/kubelet/pods/028ef312-8b95-4253-a9f7-3b92695a6e79/volumes" Aug 05 12:49:13.442023 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:13.442009 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31cc4365-94ca-403a-84d9-952af97a420a" path="/var/lib/kubelet/pods/31cc4365-94ca-403a-84d9-952af97a420a/volumes" Aug 05 12:49:24.392545 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:24.392500 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:24.511243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:24.511207 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:30.011834 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.011800 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:30.112233 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.112202 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:30.219342 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219305 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fkpss/must-gather-nt6dh"] Aug 05 12:49:30.219680 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219662 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="online" Aug 05 12:49:30.219767 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219681 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="online" Aug 05 12:49:30.219767 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219699 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="feast-apply" Aug 05 12:49:30.219767 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219707 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="feast-apply" Aug 05 12:49:30.219767 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219721 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="online" Aug 05 12:49:30.219767 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219730 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="online" Aug 05 12:49:30.219767 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219746 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="registry" Aug 05 12:49:30.219767 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219754 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="registry" Aug 05 12:49:30.219767 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219763 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="feast-init" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219771 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="feast-init" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219788 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="feast-apply" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219796 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="feast-apply" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219806 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="offline" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219814 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="offline" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219825 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="ui" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219834 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="ui" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219855 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="offline" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219863 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="offline" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219876 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="feast-init" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219885 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="feast-init" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219897 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="ui" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219907 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="ui" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.219998 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="online" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.220011 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="offline" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.220021 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="ui" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.220032 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="registry" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.220044 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="online" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.220057 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="31cc4365-94ca-403a-84d9-952af97a420a" containerName="offline" Aug 05 12:49:30.220213 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.220066 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="028ef312-8b95-4253-a9f7-3b92695a6e79" containerName="ui" Aug 05 12:49:30.223321 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.223301 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:30.225514 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.225496 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fkpss\"/\"openshift-service-ca.crt\"" Aug 05 12:49:30.225643 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.225529 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fkpss\"/\"kube-root-ca.crt\"" Aug 05 12:49:30.226304 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.226286 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fkpss\"/\"default-dockercfg-t79th\"" Aug 05 12:49:30.226398 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.226326 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:30.229854 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.229830 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fkpss/must-gather-nt6dh"] Aug 05 12:49:30.340429 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.340345 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:30.367254 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.367228 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/822ab0f4-42a4-4539-a344-c400dc821f0e-must-gather-output\") pod \"must-gather-nt6dh\" (UID: \"822ab0f4-42a4-4539-a344-c400dc821f0e\") " pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:30.367373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.367270 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn4m6\" (UniqueName: \"kubernetes.io/projected/822ab0f4-42a4-4539-a344-c400dc821f0e-kube-api-access-pn4m6\") pod \"must-gather-nt6dh\" (UID: \"822ab0f4-42a4-4539-a344-c400dc821f0e\") " pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:30.468022 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.467995 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/822ab0f4-42a4-4539-a344-c400dc821f0e-must-gather-output\") pod \"must-gather-nt6dh\" (UID: \"822ab0f4-42a4-4539-a344-c400dc821f0e\") " pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:30.468157 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.468051 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn4m6\" (UniqueName: \"kubernetes.io/projected/822ab0f4-42a4-4539-a344-c400dc821f0e-kube-api-access-pn4m6\") pod \"must-gather-nt6dh\" (UID: \"822ab0f4-42a4-4539-a344-c400dc821f0e\") " pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:30.468312 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.468295 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/822ab0f4-42a4-4539-a344-c400dc821f0e-must-gather-output\") pod \"must-gather-nt6dh\" (UID: \"822ab0f4-42a4-4539-a344-c400dc821f0e\") " pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:30.475711 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.475691 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn4m6\" (UniqueName: \"kubernetes.io/projected/822ab0f4-42a4-4539-a344-c400dc821f0e-kube-api-access-pn4m6\") pod \"must-gather-nt6dh\" (UID: \"822ab0f4-42a4-4539-a344-c400dc821f0e\") " pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:30.546624 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.546599 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:30.676769 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.676737 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fkpss/must-gather-nt6dh"] Aug 05 12:49:30.677615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.677591 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 12:49:30.816143 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:30.816111 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fkpss/must-gather-nt6dh" event={"ID":"822ab0f4-42a4-4539-a344-c400dc821f0e","Type":"ContainerStarted","Data":"e6349cde18e0a84a526486138f592338b9bfcaa4074f9d3664c4705acf4d656c"} Aug 05 12:49:36.846453 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:36.846430 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fkpss/must-gather-nt6dh" event={"ID":"822ab0f4-42a4-4539-a344-c400dc821f0e","Type":"ContainerStarted","Data":"69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4"} Aug 05 12:49:37.852623 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:37.852589 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fkpss/must-gather-nt6dh" event={"ID":"822ab0f4-42a4-4539-a344-c400dc821f0e","Type":"ContainerStarted","Data":"a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948"} Aug 05 12:49:37.876325 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:37.876278 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fkpss/must-gather-nt6dh" podStartSLOduration=1.8317688479999998 podStartE2EDuration="7.876261345s" totalImagesPullingTime="6.044492497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 12:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:49:37.874161895 +0000 UTC m=+1842.939753796" watchObservedRunningTime="2026-08-05 12:49:37.876261345 +0000 UTC m=+1842.941853243" Aug 05 12:49:46.889526 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:46.889490 2569 generic.go:356] "Generic (PLEG): container finished" podID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerID="69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4" exitCode=0 Aug 05 12:49:46.889939 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:46.889558 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fkpss/must-gather-nt6dh" event={"ID":"822ab0f4-42a4-4539-a344-c400dc821f0e","Type":"ContainerDied","Data":"69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4"} Aug 05 12:49:46.889939 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:46.889848 2569 scope.go:117] "RemoveContainer" containerID="69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4" Aug 05 12:49:47.181307 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:47.181228 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fkpss_must-gather-nt6dh_822ab0f4-42a4-4539-a344-c400dc821f0e/gather/0.log" Aug 05 12:49:47.545826 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:47.545795 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:47.697966 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:47.697921 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:47.755815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:47.755781 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:50.221230 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:50.221201 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c6j9w_a352b881-3036-4ed7-971a-22645878ebfd/global-pull-secret-syncer/0.log" Aug 05 12:49:50.364684 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:50.364653 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" Aug 05 12:49:50.492424 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:50.492355 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-178.ec2.internal_2de2a3e1acdc244afa40a51cd31b1009/haproxy/0.log" Aug 05 12:49:52.523795 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.523764 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-fkpss/must-gather-nt6dh"] Aug 05 12:49:52.524189 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.523943 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-fkpss/must-gather-nt6dh" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerName="copy" containerID="cri-o://a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948" gracePeriod=2 Aug 05 12:49:52.527853 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.527793 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-fkpss/must-gather-nt6dh"] Aug 05 12:49:52.528584 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:49:52.528383 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nt6dh\" is forbidden: User \"system:node:ip-10-0-143-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fkpss\": no relationship found between node 'ip-10-0-143-178.ec2.internal' and this object" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:52.751679 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.751659 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fkpss_must-gather-nt6dh_822ab0f4-42a4-4539-a344-c400dc821f0e/copy/0.log" Aug 05 12:49:52.753486 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.753467 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:52.755322 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:49:52.755298 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nt6dh\" is forbidden: User \"system:node:ip-10-0-143-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fkpss\": no relationship found between node 'ip-10-0-143-178.ec2.internal' and this object" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:52.848941 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.848882 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/822ab0f4-42a4-4539-a344-c400dc821f0e-must-gather-output\") pod \"822ab0f4-42a4-4539-a344-c400dc821f0e\" (UID: \"822ab0f4-42a4-4539-a344-c400dc821f0e\") " Aug 05 12:49:52.849056 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.849002 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn4m6\" (UniqueName: \"kubernetes.io/projected/822ab0f4-42a4-4539-a344-c400dc821f0e-kube-api-access-pn4m6\") pod \"822ab0f4-42a4-4539-a344-c400dc821f0e\" (UID: \"822ab0f4-42a4-4539-a344-c400dc821f0e\") " Aug 05 12:49:52.849238 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.849214 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822ab0f4-42a4-4539-a344-c400dc821f0e-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "822ab0f4-42a4-4539-a344-c400dc821f0e" (UID: "822ab0f4-42a4-4539-a344-c400dc821f0e"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 12:49:52.851043 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.851020 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/822ab0f4-42a4-4539-a344-c400dc821f0e-kube-api-access-pn4m6" (OuterVolumeSpecName: "kube-api-access-pn4m6") pod "822ab0f4-42a4-4539-a344-c400dc821f0e" (UID: "822ab0f4-42a4-4539-a344-c400dc821f0e"). InnerVolumeSpecName "kube-api-access-pn4m6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 12:49:52.915139 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.915121 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fkpss_must-gather-nt6dh_822ab0f4-42a4-4539-a344-c400dc821f0e/copy/0.log" Aug 05 12:49:52.915444 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.915424 2569 generic.go:356] "Generic (PLEG): container finished" podID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerID="a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948" exitCode=143 Aug 05 12:49:52.915512 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.915472 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:52.915560 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.915536 2569 scope.go:117] "RemoveContainer" containerID="a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948" Aug 05 12:49:52.917449 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:49:52.917422 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nt6dh\" is forbidden: User \"system:node:ip-10-0-143-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fkpss\": no relationship found between node 'ip-10-0-143-178.ec2.internal' and this object" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:52.923119 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.923027 2569 scope.go:117] "RemoveContainer" containerID="69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4" Aug 05 12:49:52.925413 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:49:52.925372 2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-nt6dh\" is forbidden: User \"system:node:ip-10-0-143-178.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fkpss\": no relationship found between node 'ip-10-0-143-178.ec2.internal' and this object" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" pod="openshift-must-gather-fkpss/must-gather-nt6dh" Aug 05 12:49:52.937187 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.937169 2569 scope.go:117] "RemoveContainer" containerID="a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948" Aug 05 12:49:52.937433 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:49:52.937414 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948\": container with ID starting with a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948 not found: ID does not exist" containerID="a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948" Aug 05 12:49:52.937505 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.937444 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948"} err="failed to get container status \"a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948\": rpc error: code = NotFound desc = could not find container \"a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948\": container with ID starting with a974c659af0aac81e866d636b43a040dc03374171c9593075e2b081a1af01948 not found: ID does not exist" Aug 05 12:49:52.937505 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.937466 2569 scope.go:117] "RemoveContainer" containerID="69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4" Aug 05 12:49:52.937712 ip-10-0-143-178 kubenswrapper[2569]: E0805 12:49:52.937693 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4\": container with ID starting with 69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4 not found: ID does not exist" containerID="69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4" Aug 05 12:49:52.937751 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.937718 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4"} err="failed to get container status \"69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4\": rpc error: code = NotFound desc = could not find container \"69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4\": container with ID starting with 69d9537092103deeb43c034f8cdbceb230f7e329728baf235a526085b029bdf4 not found: ID does not exist" Aug 05 12:49:52.950308 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.950286 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn4m6\" (UniqueName: \"kubernetes.io/projected/822ab0f4-42a4-4539-a344-c400dc821f0e-kube-api-access-pn4m6\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:52.950308 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:52.950308 2569 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/822ab0f4-42a4-4539-a344-c400dc821f0e-must-gather-output\") on node \"ip-10-0-143-178.ec2.internal\" DevicePath \"\"" Aug 05 12:49:53.132063 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:53.131982 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:53.231173 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:53.231141 2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2fjj5_61f27d7c-18bb-4986-98b1-0908ccfeec3e/konnectivity-agent/0.log" limit=1024 Aug 05 12:49:53.441815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:53.441741 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" path="/var/lib/kubelet/pods/822ab0f4-42a4-4539-a344-c400dc821f0e/volumes" Aug 05 12:49:53.982800 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:53.982773 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e567e0c8-24d3-483a-93b0-7e2233c9f693/alertmanager/0.log" Aug 05 12:49:54.027567 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.027541 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e567e0c8-24d3-483a-93b0-7e2233c9f693/config-reloader/0.log" Aug 05 12:49:54.064930 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.064909 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e567e0c8-24d3-483a-93b0-7e2233c9f693/kube-rbac-proxy-web/0.log" Aug 05 12:49:54.111656 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.111634 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e567e0c8-24d3-483a-93b0-7e2233c9f693/kube-rbac-proxy/0.log" Aug 05 12:49:54.159900 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.159877 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e567e0c8-24d3-483a-93b0-7e2233c9f693/kube-rbac-proxy-metric/0.log" Aug 05 12:49:54.193157 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.193138 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e567e0c8-24d3-483a-93b0-7e2233c9f693/prom-label-proxy/0.log" Aug 05 12:49:54.224169 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.224148 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e567e0c8-24d3-483a-93b0-7e2233c9f693/init-config-reloader/0.log" Aug 05 12:49:54.272650 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.272624 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-swmv4_b97663a6-9d83-437a-b7c7-13f5ab0445ae/cluster-monitoring-operator/0.log" Aug 05 12:49:54.302512 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.302495 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-d69mj_c70ebb7b-1a52-4f4b-b8f2-430938d63054/kube-state-metrics/0.log" Aug 05 12:49:54.328748 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.328728 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-d69mj_c70ebb7b-1a52-4f4b-b8f2-430938d63054/kube-rbac-proxy-main/0.log" Aug 05 12:49:54.352459 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.352438 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-d69mj_c70ebb7b-1a52-4f4b-b8f2-430938d63054/kube-rbac-proxy-self/0.log" Aug 05 12:49:54.381741 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.381720 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b665b4949-2rlhw_a68d94e0-b82d-435d-8cb9-3f0aee82b217/metrics-server/0.log" Aug 05 12:49:54.532083 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.532063 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lmkl_9b56f15e-126a-4abc-bdd0-3c48cd3ba988/node-exporter/0.log" Aug 05 12:49:54.562929 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.562909 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lmkl_9b56f15e-126a-4abc-bdd0-3c48cd3ba988/kube-rbac-proxy/0.log" Aug 05 12:49:54.588782 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:54.588764 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8lmkl_9b56f15e-126a-4abc-bdd0-3c48cd3ba988/init-textfile/0.log" Aug 05 12:49:55.033303 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:55.033276 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d4cbd77b7-7t9tn_8b4c9d0c-8874-443a-b6a9-f34d4b7d8131/telemeter-client/0.log" Aug 05 12:49:55.058073 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:55.058028 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d4cbd77b7-7t9tn_8b4c9d0c-8874-443a-b6a9-f34d4b7d8131/reload/0.log" Aug 05 12:49:55.084223 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:55.084203 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d4cbd77b7-7t9tn_8b4c9d0c-8874-443a-b6a9-f34d4b7d8131/kube-rbac-proxy/0.log" Aug 05 12:49:56.437870 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:56.437844 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-xhr85_eb4fa7e4-db72-4734-8a4b-2a55cc102395/networking-console-plugin/0.log" Aug 05 12:49:56.875626 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:56.875599 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/1.log" Aug 05 12:49:56.879315 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:56.879295 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-z9wlp_bbc43ec4-08bb-4c69-b4e2-94b3e60eb797/console-operator/2.log" Aug 05 12:49:57.269144 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:57.269118 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5dc6dc6d44-f5c6z_812f98d1-0dc9-4f99-b6f9-28322a514c81/console/0.log" Aug 05 12:49:57.759407 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:57.759383 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-rrnrx_7a5ecb58-108a-4caf-a322-a8850ead53a5/volume-data-source-validator/0.log" Aug 05 12:49:58.121439 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.121344 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn"] Aug 05 12:49:58.121676 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.121664 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerName="copy" Aug 05 12:49:58.121723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.121678 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerName="copy" Aug 05 12:49:58.121723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.121688 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerName="gather" Aug 05 12:49:58.121723 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.121694 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerName="gather" Aug 05 12:49:58.121833 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.121742 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerName="copy" Aug 05 12:49:58.121833 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.121753 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="822ab0f4-42a4-4539-a344-c400dc821f0e" containerName="gather" Aug 05 12:49:58.123418 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.123395 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.125508 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.125489 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qsg45\"/\"openshift-service-ca.crt\"" Aug 05 12:49:58.125508 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.125500 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qsg45\"/\"kube-root-ca.crt\"" Aug 05 12:49:58.125759 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.125741 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qsg45\"/\"default-dockercfg-skb8c\"" Aug 05 12:49:58.134653 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.134629 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn"] Aug 05 12:49:58.185342 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.185310 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-lib-modules\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.185449 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.185358 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-podres\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.185449 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.185389 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-sys\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.185526 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.185444 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-proc\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.185526 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.185486 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px7lk\" (UniqueName: \"kubernetes.io/projected/712f3167-d21f-40c0-8c51-dd92bcd624ba-kube-api-access-px7lk\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286050 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286022 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px7lk\" (UniqueName: \"kubernetes.io/projected/712f3167-d21f-40c0-8c51-dd92bcd624ba-kube-api-access-px7lk\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286157 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286058 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-lib-modules\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286157 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286085 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-podres\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286157 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286118 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-sys\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286314 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286165 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-proc\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286314 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286226 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-proc\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286314 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286231 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-sys\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286314 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286231 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-lib-modules\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.286314 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.286269 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/712f3167-d21f-40c0-8c51-dd92bcd624ba-podres\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.293195 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.293172 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px7lk\" (UniqueName: \"kubernetes.io/projected/712f3167-d21f-40c0-8c51-dd92bcd624ba-kube-api-access-px7lk\") pod \"perf-node-gather-daemonset-j6lcn\" (UID: \"712f3167-d21f-40c0-8c51-dd92bcd624ba\") " pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.433373 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.433283 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.556071 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.556038 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn"] Aug 05 12:49:58.559532 ip-10-0-143-178 kubenswrapper[2569]: W0805 12:49:58.559501 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod712f3167_d21f_40c0_8c51_dd92bcd624ba.slice/crio-d82aebeb8b2fb22e0521e902da3aded523c4807a20c7d257d5aa48573906d65b WatchSource:0}: Error finding container d82aebeb8b2fb22e0521e902da3aded523c4807a20c7d257d5aa48573906d65b: Status 404 returned error can't find the container with id d82aebeb8b2fb22e0521e902da3aded523c4807a20c7d257d5aa48573906d65b Aug 05 12:49:58.622156 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.622123 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wlqgr_5cc3ea90-8123-4429-a46e-ae3f0ab302c1/dns/0.log" Aug 05 12:49:58.641774 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.641754 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wlqgr_5cc3ea90-8123-4429-a46e-ae3f0ab302c1/kube-rbac-proxy/0.log" Aug 05 12:49:58.689060 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.688998 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j7c8w_00e242a0-2726-4da2-b46e-044a68d05c5d/dns-node-resolver/0.log" Aug 05 12:49:58.942162 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.942069 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" event={"ID":"712f3167-d21f-40c0-8c51-dd92bcd624ba","Type":"ContainerStarted","Data":"5c84380798789ec0669c6dfc135b06a0011ef669b05d1e23256fb7b90410ff27"} Aug 05 12:49:58.942162 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.942131 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:49:58.942162 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.942154 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" event={"ID":"712f3167-d21f-40c0-8c51-dd92bcd624ba","Type":"ContainerStarted","Data":"d82aebeb8b2fb22e0521e902da3aded523c4807a20c7d257d5aa48573906d65b"} Aug 05 12:49:58.958845 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:58.958797 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" podStartSLOduration=0.958783091 podStartE2EDuration="958.783091ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 12:49:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 12:49:58.956674278 +0000 UTC m=+1864.022266176" watchObservedRunningTime="2026-08-05 12:49:58.958783091 +0000 UTC m=+1864.024374989" Aug 05 12:49:59.126718 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:59.126678 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p286z_b8d4ced4-b21e-4a12-8a35-269ad385fcc6/node-ca/0.log" Aug 05 12:49:59.818818 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:49:59.818789 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cc769bc88-2vr74_d879f9c0-8e83-4585-b3f9-9b1f72fc3e4e/router/0.log" Aug 05 12:50:00.175423 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:00.175348 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jtxvw_f71fe19e-7bc3-497c-a52e-f21eee789121/serve-healthcheck-canary/0.log" Aug 05 12:50:00.517358 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:00.517330 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-7qgvq_dd67cf94-77ba-4851-8abb-2981237ae617/insights-operator/0.log" Aug 05 12:50:00.517687 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:00.517671 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-7qgvq_dd67cf94-77ba-4851-8abb-2981237ae617/insights-operator/1.log" Aug 05 12:50:00.595947 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:00.595927 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shpvc_ab686670-c502-4b04-aaad-6cb41a462119/kube-rbac-proxy/0.log" Aug 05 12:50:00.618448 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:00.618427 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shpvc_ab686670-c502-4b04-aaad-6cb41a462119/exporter/0.log" Aug 05 12:50:00.638475 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:00.638439 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shpvc_ab686670-c502-4b04-aaad-6cb41a462119/extractor/0.log" Aug 05 12:50:04.955886 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:04.955857 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qsg45/perf-node-gather-daemonset-j6lcn" Aug 05 12:50:04.973007 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:04.972984 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-s8grw_1e3ef6c1-6fc9-4467-ae88-65c7b7b24436/kube-storage-version-migrator-operator/1.log" Aug 05 12:50:04.973742 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:04.973727 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-s8grw_1e3ef6c1-6fc9-4467-ae88-65c7b7b24436/kube-storage-version-migrator-operator/0.log" Aug 05 12:50:06.032606 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.032573 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4ll9_73786aea-c106-4636-9696-a5ec46bfeb24/kube-multus-additional-cni-plugins/0.log" Aug 05 12:50:06.053597 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.053568 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4ll9_73786aea-c106-4636-9696-a5ec46bfeb24/egress-router-binary-copy/0.log" Aug 05 12:50:06.074564 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.074542 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4ll9_73786aea-c106-4636-9696-a5ec46bfeb24/cni-plugins/0.log" Aug 05 12:50:06.093842 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.093823 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4ll9_73786aea-c106-4636-9696-a5ec46bfeb24/bond-cni-plugin/0.log" Aug 05 12:50:06.112876 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.112862 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4ll9_73786aea-c106-4636-9696-a5ec46bfeb24/routeoverride-cni/0.log" Aug 05 12:50:06.133243 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.133223 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4ll9_73786aea-c106-4636-9696-a5ec46bfeb24/whereabouts-cni-bincopy/0.log" Aug 05 12:50:06.154445 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.154420 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w4ll9_73786aea-c106-4636-9696-a5ec46bfeb24/whereabouts-cni/0.log" Aug 05 12:50:06.235556 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.235533 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tp9z5_d6a8082e-295e-4dc6-ae1e-35ce58a50e8f/kube-multus/0.log" Aug 05 12:50:06.300628 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.300566 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vsh45_bccda255-92ed-4f4c-b52f-f3e7c6a639bf/network-metrics-daemon/0.log" Aug 05 12:50:06.320689 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:06.320668 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vsh45_bccda255-92ed-4f4c-b52f-f3e7c6a639bf/kube-rbac-proxy/0.log" Aug 05 12:50:07.172304 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:07.172263 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k2j42_e4155f61-9591-437e-a077-b395af2f689e/ovn-controller/0.log" Aug 05 12:50:07.193273 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:07.193252 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k2j42_e4155f61-9591-437e-a077-b395af2f689e/ovn-acl-logging/0.log" Aug 05 12:50:07.217273 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:07.217253 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k2j42_e4155f61-9591-437e-a077-b395af2f689e/kube-rbac-proxy-node/0.log" Aug 05 12:50:07.238703 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:07.238686 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k2j42_e4155f61-9591-437e-a077-b395af2f689e/kube-rbac-proxy-ovn-metrics/0.log" Aug 05 12:50:07.258615 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:07.258589 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k2j42_e4155f61-9591-437e-a077-b395af2f689e/northd/0.log" Aug 05 12:50:07.277717 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:07.277659 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k2j42_e4155f61-9591-437e-a077-b395af2f689e/nbdb/0.log" Aug 05 12:50:07.300815 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:07.300798 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k2j42_e4155f61-9591-437e-a077-b395af2f689e/sbdb/0.log" Aug 05 12:50:07.339975 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:07.339930 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k2j42_e4155f61-9591-437e-a077-b395af2f689e/ovnkube-controller/0.log" Aug 05 12:50:08.636764 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:08.636735 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-65bdbbff45-dq9dj_552e4b27-d3d2-4ae2-9001-2be16d09b667/check-endpoints/0.log" Aug 05 12:50:08.684792 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:08.684764 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-z29nk_bb6cf0be-ed34-4473-b64d-d12bf3212eae/network-check-target-container/0.log" Aug 05 12:50:09.580625 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:09.580598 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-b27jc_d3391f80-10ec-4a2c-abe5-a50d4cd2bc24/iptables-alerter/0.log" Aug 05 12:50:10.166310 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:10.166280 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-gg2w9_b01b0a6a-4ac3-48f9-aa3d-2ab421bdbd9d/tuned/0.log" Aug 05 12:50:11.850560 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:11.850531 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-559cbf5745-fxmhb_cdd81e42-4928-48b5-9e19-296cb4774bef/cluster-samples-operator/0.log" Aug 05 12:50:11.865368 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:11.865348 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-559cbf5745-fxmhb_cdd81e42-4928-48b5-9e19-296cb4774bef/cluster-samples-operator-watch/0.log" Aug 05 12:50:12.746051 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:12.746021 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-784f97bcf6-lxj6z_4352fed2-2dde-4ff6-9cb7-998b3fded2e7/service-ca-operator/1.log" Aug 05 12:50:12.747258 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:12.747235 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-784f97bcf6-lxj6z_4352fed2-2dde-4ff6-9cb7-998b3fded2e7/service-ca-operator/0.log" Aug 05 12:50:13.498591 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:13.498518 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-www5s_6663f904-8a23-4be1-9d95-6c7b4585a35a/csi-driver/0.log" Aug 05 12:50:13.535187 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:13.535163 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-www5s_6663f904-8a23-4be1-9d95-6c7b4585a35a/csi-node-driver-registrar/0.log" Aug 05 12:50:13.554628 ip-10-0-143-178 kubenswrapper[2569]: I0805 12:50:13.554606 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-www5s_6663f904-8a23-4be1-9d95-6c7b4585a35a/csi-liveness-probe/0.log"